[llvm-dev] LLVM_DYLIB and CLANG_DYLIB with MSVC

[Cristian Adam via llvm-dev]

On 03/06/2021 23:09, Martin Storsjö wrote:

> On Sun, 30 May 2021, Cristian Adam via llvm-dev wrote:
>
>> Due to the nature of MSVC regarding default visibility of symbols 
>> (hidden by
>> default, whereas MinGW has visible by default), one needs to generate 
>> a .def
>> file with the symbols needed to be exported.
>>
>> This is done already in two cases for LLVM_BUILD_LLVM_C_DYLIB
>> (llvm/tools/llvm-shlib/gen-msvc-exports.py) and for
>> LLVM_EXPORT_SYMBOLS_FOR_PLUGINS (llvm/utils/extract_symbols.py).
>>
>> I've put together a patch that enables LLVM_DYLIB and CLANG_DYLIB for 
>> MSVC.
>>
>> I tested with clang-cl from the official Clang 12 x64 Windows binary
>> release:
>>
>>  *  Normal build: 1,42 GB
>>  *  shlib build: 536 MB
>>
>> The shlib release build compiled and linked fine with LLVM.dll and
>> clang-cpp.dll, unfortunately it crashes at runtime.
>
> Without digging into the scripts, I have one hunch:
>
> Does the def generator script differentiate between code and data 
> symbols?
>
No. The CMake script just calls llvm-nm, exports the symbols and filters 
some out.

llvm-nm needs to exist beforehand. I tried using dumpbin, but it's 
slower than llvm-nm. CMake's __create_def also works, but it's not 
exporting all symbols needed to proper link.

> For the cases where accessing a data symbol from another DLL, the 
> caller would have to have seen a declaration with the dllimport 
> attribute. For functions, it doesn't matter (it just does an extra hop 
> via the import thunk), but for data variables it matters. If the def 
> file would have proper DATA annotations for such symbols, you would 
> end up with linker errors (where you'd have an undefined reference to 
> dataSymbol, where the import library only provides __imp_dataSymbol).
>
> This is fixed up by the autoimport feature when linking in mingw mode 
> (which, in general, requires you to link against the mingw runtime 
> too); for cases where the caller references dataSymbol but you only 
> have __imp_dataSymbol available, the linker adds an entry to a list of 
> pseudo relocations, which the mingw runtime handles when loaded, which 
> then maps sections as writable and patches up the addresses to where 
> they are located in another DLL.
>
> So to avoid this, we would either need to actually provide proper 
> dllimport declarations at least for all data symbols, or avoid 
> cross-DLL data accesses (by using e.g. accessor functions instead).
>
>
According to 
https://docs.microsoft.com/en-us/cpp/build/reference/exports?view=msvc-160 
<https://docs.microsoft.com/en-us/cpp/build/reference/exports?view=msvc-160> 
you need to:

    /When you export a variable from a DLL by using a .DEF file, you do
    not have to specify __declspec(dllexport) on the variable. However,
    in any file that uses the DLL, you must still use
    __declspec(dllimport) on the declaration of data.//
    /

Then tested it out in a small project 
https://github.com/cristianadam/test-dll-def/ 
<https://github.com/cristianadam/test-dll-def/>.

Then proceeded to add dllimport declarations for 
llvm::cl::TopLevelSubCommand and llvm::cl::AllSubCommands as seen in the 
updated patch:
https://github.com/cristianadam/llvm-project/commit/56ecad41992bd9345702fccaf3805ab186dca25c 
<https://github.com/cristianadam/llvm-project/commit/56ecad41992bd9345702fccaf3805ab186dca25c>

Now llvm-nm doesn't crash anymore!

Adding the dllimport declaration for the exported data symbols should be 
less work than doing proper dllimport declaration for everything that 
uses LLVM.dll andclang-cpp.dll. Now I /just /need to find out which data 
variables I need to update.

Thank you!

Cheers,
Cristian.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210605/1d207d76/attachment.html>