[llvm-dev] LLVM_DYLIB and CLANG_DYLIB with MSVC

Thu Jun 3 14:43:11 PDT 2021

On 2021-06-03, Reid Kleckner wrote:
>I agree, for the reasons you outline, LLVM needs to do more to support
>producing DLLs on Windows. However, I don't really have time to dig into
>your proposal and the issues you are running into.
>
>Personally, I think LLVM should move away from that .def file generation
>python script, and towards source-level annotations. It would allow us to
>use fvisibility=hidden in the shared library build on other platforms. That
>would greatly reduce the number of dynamic symbols in LLVM, which I
>understand is desirable. Today we used Bsymbolic-functions, so we may have
>already captured most of the benefits of fvisibility=hidden, but hidden
>visibility is always better for performance and binary size.

Yes, for Clang, the benefits are mostly captured with just -Bsymbolic-functions.

For GCC -fno-semantic-interposition is needed to enable interprocedural
optimizations for -fPIC compiles (https://reviews.llvm.org/D102453)

>I put out this alternative proposal mainly to see if there is any
>enthusiasm for it. If not, you are welcome to continue forward with our
>existing solutions. But if there is broad interest in adding API
>annotations to LLVM, I think that would be a better way to go long term.

I think explicit annotations make sense. I think most large-scale
projects considering ELF/Windows portability are doing this and
llvm-project is an unfortunate outlier.

If we add annotations (I persoanlly favor it), there will be churn to
llvm/include/llvm/**/*.h header files.

Moreover, as is, almost every function defined in llvm/lib/A/*.cpp is
exported to llvm/include/llvm/A/*.h if it is used by another library
llvm/lib/B.  Every cross-lib API is public. We don't do a good job
making clear what are internal and what are public (and what are
stabler and what are less).

>On Sun, May 30, 2021 at 7:17 AM Cristian Adam via llvm-dev <
>llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> Clang 12 can be configured on Windows with MinGW (either GNU or LLVM) with
>> the following CMake parameters:
>>
>>    - LLVM_BUILD_LLVM_DYLIB=ON
>>    - LLVM_LINK_LLVM_DYLIB=ON
>>    - CLANG_LINK_CLANG_DYLIB=ON
>>
>> which has some effect on the binary size of the build.
>>
>> I configured the llvm-project with the following parameters:
>>
>>    - CMAKE_BUILD_TYPE=Release
>>    - LLVM_TARGETS_TO_BUILD=X86
>>    - LLVM_ENABLE_PROJECTS=clang;clang-tools-extra
>>
>> The installed (stripped) build of Clang 12 with llvm-mingw release 12.0.0
>> <https://github.com/mstorsjo/llvm-mingw/releases/tag/20210423> resulted
>> in:
>>
>>    - Normal build: 1,76 GB
>>    - shlib build: 481 MB
>>
>> Due to the nature of MSVC regarding default visibility of symbols (hidden
>> by default, whereas MinGW has visible by default), one needs to generate a
>> .def file with the symbols needed to be exported.
>>
>> This is done already in two cases for LLVM_BUILD_LLVM_C_DYLIB (llvm/tools/llvm-shlib/gen-msvc-exports.py)
>> and for LLVM_EXPORT_SYMBOLS_FOR_PLUGINS (llvm/utils/extract_symbols.py).
>>
>> I've put together a patch
>> <https://github.com/cristianadam/llvm-project/commit/3a3b8a7df17a49ba7c0153b0c9a7ee25705ede46>
>> that enables LLVM_DYLIB and CLANG_DYLIB for MSVC.
>>
>> I tested with clang-cl from the official Clang 12 x64 Windows binary
>> release:
>>
>>    - Normal build: 1,42 GB
>>    - shlib build: 536 MB
>>
>> The shlib release build compiled and linked fine with LLVM.dll and
>> clang-cpp.dll, unfortunately it crashes at runtime. For example llvm-nm:
>>
>> $ llvm-nm
>> PLEASE submit a bug report to https://bugs.llvm.org/ and include the
>> crash backtrace.
>> Stack dump:
>> 0.      Program arguments: llvm-nm
>> #0 0x00007ffd32807d43 llvm::StringMap<llvm::cl::Option
>> *,llvm::MallocAllocator>::begin
>> C:\Projects\llvm-project\repo\llvm\include\llvm\ADT\StringMap.h:204:0
>> #1 0x00007ffd32807d43 llvm::cl::HideUnrelatedOptions(class
>> llvm::cl::OptionCategory &, class llvm::cl::SubCommand &)
>> C:\Projects\llvm-project\repo\llvm\lib\Support\CommandLine.cpp:2589:0
>> #2 0x00007ff689df2b13 llvm::StringRef::StringRef
>> C:\Projects\llvm-project\repo\llvm\include\llvm\ADT\StringRef.h:107:0
>> #3 0x00007ff689df2b13 main
>> C:\Projects\llvm-project\repo\llvm\tools\llvm-nm\llvm-nm.cpp:2232:0
>> #4 0x00007ff689e26d04 invoke_main
>> D:\agent\_work\10\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78:0
>> #5 0x00007ff689e26d04 __scrt_common_main_seh
>> D:\agent\_work\10\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
>> #6 0x00007ffd9a7f7034 (C:\Windows\System32\KERNEL32.DLL+0x17034)
>> #7 0x00007ffd9b742651 (C:\Windows\SYSTEM32\ntdll.dll+0x52651)
>>
>> This crash is due to llvm::cl::HideUnrelatedOptions which accesses
>> TopLevelSubCommand, which is defined as:
>> extern ManagedStatic<SubCommand> TopLevelSubCommand;
>>
>> The MSVC 2019 build behaves in the same way as the clang-cl build.
>>
>> I have tried building without LLVM_ENABLE_THREADS, or by linking to the
>> CRT statically LLVM_USE_CRT_RELEASE=MT, didn't help.
>>
>> The MSVC 2019 build sizes were:
>>
>>    - Normal build: 1,74 GB
>>    - shlib build: 949 MB
>>
>> I would appreciate any help in getting the shlib build running. It works
>> fine with llvm-mingw, I think it should also work with clang-cl / cl.
>>
>> Cheers,
>> Cristian.
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>