[cfe-dev] Big Windows Clang DLL

[John Thompson]

Hi,

I've been doing some experimenting to get a "big" DLL version of Clang
working. This involves changes to LLVM as well as Clang, and might apply to
doing a "big" DLL version of LLVM too (if that makes sense), so I'm posting
this to both cfe-dev and llvmdev.

Basically, I want to find out if this mechanism is acceptable, or what
alternative I should pursue.

I have a minimal version partly working, with just enough symbols exported
so I can build a clang executable against the DLL, so I thought it would be
the right time to open it up for discussion and review.

The scheme is meant to satisfy the following design goals/constraints:

1. Create a single Windows DLL including the main Clang and LLVM modules
needed for a compiler and other tools, such that it's the only module you
need to link against (from the Clang/LLVM project).

2. Preserve the current module organization and build, leaving the
static-library and other platform's shared library versions as they are.
(This leaves open the option of creating per-module Windows DLLS in the
future.)

3. Enable Clang plug-ins to share the code and data space with the Clang or
other tool executables.

4. Make the "big" DLL version optional by means of cmake arguments given
upon running cmake. (I'm not sure of the criteria for deciding whether to
make it the default for Windows or not.)

These are the build changes, with respect to the cmake build tool:

1. A "BUILD_LLVM_DLL" configuration flag is added to the cmake build, such
that specifying "-DBUILD_LLVM_DLL=ON" in the cmake command line will enable
the big-DLL build. By default it is off.

2. Factor out the common portions of the "add_clang_library" into
"add_clang_library_common".

3. Modify the cmake macro "add_clang_library" to collect the source file
full path strings in a new target property.

4. Add a new "add_clang_dll" cmake macro for creating the big DLL. This
macro operates like "add_clang_library", but always builds a DLL target, and
doesn't collect the source paths. It also passes "LLVM_USE_DLL" and
"LLVM_EXPORTS" defines to the compiler, to enable the symbol exports to be
described later.

5. Create a new "ClangDLL" project under tools/clang/lib for creating the
big DLL, which uses the new "add_clang_dll" macro.

6. Add new "set_clang_components" and "set_clang_llvm_components" cmake
macros for use in conjunction with "add_clang_executable" to collect the
Clang and LLVM components, as opposed to setting "LLVM_USED_LIBS" and
"LLVM_LINK_COMPONENTS" directory. (These macros will substitute the big DLL
for the component libraries.)

7. Modify "add_clang_executable" to pass an "LLVM_USE_DLL" define to the
compiler if the big-DLL version is enabled. This define enables the symbol
imports/exports to be described later.

8. Update the clang executable, interpreter, and PrintFunctionNames projects
to use the new and revised cmake macros.

The source changes mainly consist of adding "(moduleNameUppercase)_LINKAGE"
macro instances to the declarations of the classes, functions, and other
symbols to be exported/imported, and new header files that define these
macros. These changes can be summarized as follows:

1. A new "(moduleName)Common.h" file is added to each LLVM and Clang module
include directory. This file defines a "(moduleNameUppercase)_LINKAGE" macro
for that component module. This macro expands to nothing for the non-big-DLL
version ("LLVM_USE_DLL" is not defined to the compiler), or to the
LLVM_EXPORT macro when building the DLL ("Clang_EXPORTS" or "LLVM_EXPORTS"
or "(moduleName)_EXPORTS" is defined), or to the LLVM_IMPORT macro when
building against the DLL ("LLVM_USE_DLL" is defined). This include file will
be included by any headers in the component needing the
"(moduleNameUppercase)_LINKAGE" macro.

2. A new "LLVMCommon.h" file is added to the include/llvm directory under
the root to define "LLVM_EXPORT" and "LLVM_IMPORT" macros that wrap the
Visual C++ "__declspec(dllexport)" and "__declspec(dllimport)" directives,
as well as to define the "LLVMCORE_LINKAGE" macro. This file will be
included by all the "(moduleName)Common.h" files for the LLVM components.

3. A new "ClangCommon.h" file is added to the tools/clang/include/clang
directory. It just includes the "LLVMCommon.h" file, and will be included by
the "(moduleName)Common.h" files for the Clang components.

In my experimental version, I added most of the "(moduleName)Common.h"
files, but I only added the "(moduleNameUppercase)_LINKAGE" macro instance
to some of the symbols, at least the ones needed by the Clang driver and
interpreter to link.

Caveats:

1. The ALL_BUILD project build breaks on some missing .inc files from the
X86 target project. I think this has to do with an inherent dependency
problem in the cmake build. Even though the "(target)CodeGenTable_gen"
project dependency is added to the DLL, the .inc files are not created
before the DLL link. I will look into this. A rebuild of the "clang" project
does work, as that was the focus of my experiment. The interpreter might
build too, as it did earlier.

2. While I seem to be able to run the clang executable okay, on my machine
the clang tests don't run. They don't run for the clean tree either, so
either my machine setup is broken, or the tests in general are broken on
Windows. I haven't figured out the problem yet. I'll be looking into this
shortly, or if you know of a problem in general, please let me know.

3. As I mentioned, I only exported some of the symbols, those needed to get
the clang executables to build. A big edit (manual or automated) will be
needed to add the "(moduleNameUppercase)_LINKAGE" macros to the rest of the
symbols, and any needed "(moduleName)Common.h" headers not yet created.

4. I only enabled the X86 target in LLVM (the default for the Windows build
in general), for the purpose of this experiment. The other targets will need
to be done as well.

5. I tried running the PrintFunctionNames plugin, but there's still a
disconnect between the registry referenced by the plugin and the one in the
big DLL. I didn't look into it deep enough yet, but I'm guessing it has
something to do with the template mechanism used not being exported. I'm not
even sure how to export it. But I think the better solution is to switch to
the simplified plugin mechanism I provided a patch for in a previous post
(and which I'm still waiting for a review fornudge-nudge).

6. In building on Windows with the big DLL mechanism enabled, there are a
lot of "class 'someTemplateClass<_Ty>' needs to have dll-interface to be
used by clients of class 'someClass'" warnings. I haven't looked into how to
best fix of these yet.

7. These were somewhat hasty edits, but I verified that it still builds for
Windows with the big-DLL option turned off. Sorry if I missed something in
this write-up.

I've created two patch files with my current changes. Because they are so
big I'm not including them as an enclosure, but you can get them here:
http://www.jtlanguage.com/xfer/dll_llvm_patch.txt
http://www.jtlanguage.com/xfer/dll_clang_patch.txt

You can search on "CMakeLists.txt" and ".cmake" to see the cmake file
changes, or "Common.h" to see the new headers. Most of the rest of the
changes are just the adding of the "(moduleNameUppercase)_LINKAGE" macros to
the class and function definitions.
Again, basically, I'm hoping to find out if this mechanism is acceptable, or
what alternative I should pursue. A timely response would be appreciated, as
I'm pretty much available to work on this full time (for now).

Thanks.

-John

-- 
John Thompson
John.Thompson.JTSoftware at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20110425/7e79f16d/attachment.html>