[ros-dev] [ros-diffs] [sir_richard] 48475: (huge bunch of text)
Sven Barth
pascaldragon at googlemail.com
Sun Aug 8 10:20:23 UTC 2010
- Previous message: [ros-dev] [ros-diffs] [sir_richard] 48475: [KERNEL32]: While working on the CMAKE branch, Amine and myself discovered a rather serious issue in kernel32 (and perhaps other libraries as well). Unlike rbuild, CMake does not allow you to export non-existant DLL functions (try it: add "poopyhead" in kernel32's exports under RBuild, and will it export "poopyhead", God knowing what that will actually link to). As an additional feature on top of the "allow non-existing functions to be exported" "feature", because rbuild generates and links STDCALL function names without the proper decoration (vs. enforcing decoration at linking, but only removing it at export-time), this allows the definition (as an export) of a STDCALL function that is completely different from the actual function itself. For example, the 5-parameter Foo function is normally Foo at 20, while the 3-parameter Foo function woudl be Foo at 12. Linking one against the other would fail (say, 2 parameters were added to Foo in a newer version). However, under RBUILD, both of these would appear as "Foo", and the linker/compiler would happilly connect the caller of Foo at 3 (that has pushed 3 parameters) to the receiving side of Foo at 5 (that is about to pop 5 parameters). Even -if- decorations WERE to be applied, Foo at 12 would STILL succeed, because of the first feature, which would enable the export of Foo at 12 even though no such function exist. In a further, bizare, twist of fate, the behavior of this RBUILD "feature", when the target function is not found, is to link the exported DLL TO ITSELF. Therefore, one can see how, previously to this patch, kernel32.dll would import a dozen functions from itself (all the non-existing functions). To really seal the deal, the behavior of exported functions used by kernel32, but that are actually forwarded to another DLL deserves a special mention. GetLastError, for example, merely forwards to RtlGetLastWin32Error, so it is normal behavior to use a #define in the C code so that all internal calls to the function are routed correctly. This did not happen, so instead, kernel32 tried importing/linking/exporting GetLastError, but this symbol is not found in the binary, because it is only a forwarder. This caused kernel32 to import from itself (the behavior when an exported symbol is not found). When importing from itself, the loader would now find the _forwarded_ for GetLastError, and correctly link with ntdll. What should be a one-liner of assembly (inline TEB access) thus became a triple-call indirection (GetLastError at 0->StubLastError at 0->__impGetLastError at 0->__impRtlGetLastWin32Error->RtlGetLastWin32Error. While analyzing these issues, we also realized a strange macro SetLastErrorByStatus that manually tried to perform what there already exists a function for: RtlSetLastNtStatusFromWin32Error. And, in an exciting coda, we also realized that our Server 2003 Kernel32 exports more than a dozen Windows 95 APIs, through an auto-stub generation mechanism within winebuild, that gets linked as an object behind the scenes. [KERNEL32]: Removed all Win95 exports, cleaned up exports. [KERNEL32]: Fixed up set/get error macros by making them inline and/or calling the correct ntdll function. [KERNEL32]: Removed bizare calls to Wine-internal/specific APIs from our core Win32 DLL. [KERNEL32]: Wrote stubs for all functions which should be exported, and set the correct number of parameters for them. [KERNEL32]: Kernel32 is smaller, loads faster, does not export Windows 95 functions, does not export non-existing functions, and does not import from itself anymore. Note: This is one of the many failings of RBUILD the CMAKE system has helped us discover. I believe these issues are serious enough to warrant an immediate sync with trunk, but rest assured, there are many more completely broken, infinitely-regressing things that we discovered while switching to CMAKE.
- Next message: [ros-dev] [ros-diffs] [sir_richard] 48475: [KERNEL32]: While working on the CMAKE branch, Amine and myself discovered a rather serious issue in kernel32 (and perhaps other libraries as well). Unlike rbuild, CMake does not allow you to export non-existant DLL functions (try it: add "poopyhead" in kernel32's exports under RBuild, and will it export "poopyhead", God knowing what that will actually link to). As an additional feature on top of the "allow non-existing functions to be exported" "feature", because rbuild generates and links STDCALL function names without the proper decoration (vs. enforcing decoration at linking, but only removing it at export-time), this allows the definition (as an export) of a STDCALL function that is completely different from the actual function itself. For example, the 5-parameter Foo function is normally Foo at 20, while the 3-parameter Foo function woudl be Foo at 12. Linking one against the other would fail (say, 2 parameters were added to Foo in a newer version). However, under RBUILD, both of these would appear as "Foo", and the linker/compiler would happilly connect the caller of Foo at 3 (that has pushed 3 parameters) to the receiving side of Foo at 5 (that is about to pop 5 parameters). Even -if- decorations WERE to be applied, Foo at 12 would STILL succeed, because of the first feature, which would enable the export of Foo at 12 even though no such function exist. In a further, bizare, twist of fate, the behavior of this RBUILD "feature", when the target function is not found, is to link the exported DLL TO ITSELF. Therefore, one can see how, previously to this patch, kernel32.dll would import a dozen functions from itself (all the non-existing functions). To really seal the deal, the behavior of exported functions used by kernel32, but that are actually forwarded to another DLL deserves a special mention. GetLastError, for example, merely forwards to RtlGetLastWin32Error, so it is normal behavior to use a #define in the C code so that all internal calls to the function are routed correctly. This did not happen, so instead, kernel32 tried importing/linking/exporting GetLastError, but this symbol is not found in the binary, because it is only a forwarder. This caused kernel32 to import from itself (the behavior when an exported symbol is not found). When importing from itself, the loader would now find the _forwarded_ for GetLastError, and correctly link with ntdll. What should be a one-liner of assembly (inline TEB access) thus became a triple-call indirection (GetLastError at 0->StubLastError at 0->__impGetLastError at 0->__impRtlGetLastWin32Error->RtlGetLastWin32Error. While analyzing these issues, we also realized a strange macro SetLastErrorByStatus that manually tried to perform what there already exists a function for: RtlSetLastNtStatusFromWin32Error. And, in an exciting coda, we also realized that our Server 2003 Kernel32 exports more than a dozen Windows 95 APIs, through an auto-stub generation mechanism within winebuild, that gets linked as an object behind the scenes. [KERNEL32]: Removed all Win95 exports, cleaned up exports. [KERNEL32]: Fixed up set/get error macros by making them inline and/or calling the correct ntdll function. [KERNEL32]: Removed bizare calls to Wine-internal/specific APIs from our core Win32 DLL. [KERNEL32]: Wrote stubs for all functions which should be exported, and set the correct number of parameters for them. [KERNEL32]: Kernel32 is smaller, loads faster, does not export Windows 95 functions, does not export non-existing functions, and does not import from itself anymore. Note: This is one of the many failings of RBUILD the CMAKE system has helped us discover. I believe these issues are serious enough to warrant an immediate sync with trunk, but rest assured, there are many more completely broken, infinitely-regressing things that we discovered while switching to CMAKE.
- Messages sorted by:
[ date ]
[ thread ]
[ subject ]
[ author ]
Hi!
Just a little question regarding commit-mails: Does the subject NEED to
be the complete svn log message? This mail's subject line was that long
that I'm not able to view the message itself in Thunderbird without
writing a reply so that I can see the quoted message body.
Couldn't the component/system/process that converts the commits into
mails for the diffs mailing list truncate the subject line? As far as I
see it the log message is put into the mail's body as well and it's also
nicer to view there because of line breaks. :)
Regards,
Sven
- Previous message: [ros-dev] [ros-diffs] [sir_richard] 48475: [KERNEL32]: While working on the CMAKE branch, Amine and myself discovered a rather serious issue in kernel32 (and perhaps other libraries as well). Unlike rbuild, CMake does not allow you to export non-existant DLL functions (try it: add "poopyhead" in kernel32's exports under RBuild, and will it export "poopyhead", God knowing what that will actually link to). As an additional feature on top of the "allow non-existing functions to be exported" "feature", because rbuild generates and links STDCALL function names without the proper decoration (vs. enforcing decoration at linking, but only removing it at export-time), this allows the definition (as an export) of a STDCALL function that is completely different from the actual function itself. For example, the 5-parameter Foo function is normally Foo at 20, while the 3-parameter Foo function woudl be Foo at 12. Linking one against the other would fail (say, 2 parameters were added to Foo in a newer version). However, under RBUILD, both of these would appear as "Foo", and the linker/compiler would happilly connect the caller of Foo at 3 (that has pushed 3 parameters) to the receiving side of Foo at 5 (that is about to pop 5 parameters). Even -if- decorations WERE to be applied, Foo at 12 would STILL succeed, because of the first feature, which would enable the export of Foo at 12 even though no such function exist. In a further, bizare, twist of fate, the behavior of this RBUILD "feature", when the target function is not found, is to link the exported DLL TO ITSELF. Therefore, one can see how, previously to this patch, kernel32.dll would import a dozen functions from itself (all the non-existing functions). To really seal the deal, the behavior of exported functions used by kernel32, but that are actually forwarded to another DLL deserves a special mention. GetLastError, for example, merely forwards to RtlGetLastWin32Error, so it is normal behavior to use a #define in the C code so that all internal calls to the function are routed correctly. This did not happen, so instead, kernel32 tried importing/linking/exporting GetLastError, but this symbol is not found in the binary, because it is only a forwarder. This caused kernel32 to import from itself (the behavior when an exported symbol is not found). When importing from itself, the loader would now find the _forwarded_ for GetLastError, and correctly link with ntdll. What should be a one-liner of assembly (inline TEB access) thus became a triple-call indirection (GetLastError at 0->StubLastError at 0->__impGetLastError at 0->__impRtlGetLastWin32Error->RtlGetLastWin32Error. While analyzing these issues, we also realized a strange macro SetLastErrorByStatus that manually tried to perform what there already exists a function for: RtlSetLastNtStatusFromWin32Error. And, in an exciting coda, we also realized that our Server 2003 Kernel32 exports more than a dozen Windows 95 APIs, through an auto-stub generation mechanism within winebuild, that gets linked as an object behind the scenes. [KERNEL32]: Removed all Win95 exports, cleaned up exports. [KERNEL32]: Fixed up set/get error macros by making them inline and/or calling the correct ntdll function. [KERNEL32]: Removed bizare calls to Wine-internal/specific APIs from our core Win32 DLL. [KERNEL32]: Wrote stubs for all functions which should be exported, and set the correct number of parameters for them. [KERNEL32]: Kernel32 is smaller, loads faster, does not export Windows 95 functions, does not export non-existing functions, and does not import from itself anymore. Note: This is one of the many failings of RBUILD the CMAKE system has helped us discover. I believe these issues are serious enough to warrant an immediate sync with trunk, but rest assured, there are many more completely broken, infinitely-regressing things that we discovered while switching to CMAKE.
- Next message: [ros-dev] [ros-diffs] [sir_richard] 48475: [KERNEL32]: While working on the CMAKE branch, Amine and myself discovered a rather serious issue in kernel32 (and perhaps other libraries as well). Unlike rbuild, CMake does not allow you to export non-existant DLL functions (try it: add "poopyhead" in kernel32's exports under RBuild, and will it export "poopyhead", God knowing what that will actually link to). As an additional feature on top of the "allow non-existing functions to be exported" "feature", because rbuild generates and links STDCALL function names without the proper decoration (vs. enforcing decoration at linking, but only removing it at export-time), this allows the definition (as an export) of a STDCALL function that is completely different from the actual function itself. For example, the 5-parameter Foo function is normally Foo at 20, while the 3-parameter Foo function woudl be Foo at 12. Linking one against the other would fail (say, 2 parameters were added to Foo in a newer version). However, under RBUILD, both of these would appear as "Foo", and the linker/compiler would happilly connect the caller of Foo at 3 (that has pushed 3 parameters) to the receiving side of Foo at 5 (that is about to pop 5 parameters). Even -if- decorations WERE to be applied, Foo at 12 would STILL succeed, because of the first feature, which would enable the export of Foo at 12 even though no such function exist. In a further, bizare, twist of fate, the behavior of this RBUILD "feature", when the target function is not found, is to link the exported DLL TO ITSELF. Therefore, one can see how, previously to this patch, kernel32.dll would import a dozen functions from itself (all the non-existing functions). To really seal the deal, the behavior of exported functions used by kernel32, but that are actually forwarded to another DLL deserves a special mention. GetLastError, for example, merely forwards to RtlGetLastWin32Error, so it is normal behavior to use a #define in the C code so that all internal calls to the function are routed correctly. This did not happen, so instead, kernel32 tried importing/linking/exporting GetLastError, but this symbol is not found in the binary, because it is only a forwarder. This caused kernel32 to import from itself (the behavior when an exported symbol is not found). When importing from itself, the loader would now find the _forwarded_ for GetLastError, and correctly link with ntdll. What should be a one-liner of assembly (inline TEB access) thus became a triple-call indirection (GetLastError at 0->StubLastError at 0->__impGetLastError at 0->__impRtlGetLastWin32Error->RtlGetLastWin32Error. While analyzing these issues, we also realized a strange macro SetLastErrorByStatus that manually tried to perform what there already exists a function for: RtlSetLastNtStatusFromWin32Error. And, in an exciting coda, we also realized that our Server 2003 Kernel32 exports more than a dozen Windows 95 APIs, through an auto-stub generation mechanism within winebuild, that gets linked as an object behind the scenes. [KERNEL32]: Removed all Win95 exports, cleaned up exports. [KERNEL32]: Fixed up set/get error macros by making them inline and/or calling the correct ntdll function. [KERNEL32]: Removed bizare calls to Wine-internal/specific APIs from our core Win32 DLL. [KERNEL32]: Wrote stubs for all functions which should be exported, and set the correct number of parameters for them. [KERNEL32]: Kernel32 is smaller, loads faster, does not export Windows 95 functions, does not export non-existing functions, and does not import from itself anymore. Note: This is one of the many failings of RBUILD the CMAKE system has helped us discover. I believe these issues are serious enough to warrant an immediate sync with trunk, but rest assured, there are many more completely broken, infinitely-regressing things that we discovered while switching to CMAKE.
- Messages sorted by:
[ date ]
[ thread ]
[ subject ]
[ author ]
More information about the Ros-dev
mailing list