[PATCH] Implement a sane plugin API for clang

[Matthew Curtis]

On 3/20/2013 8:17 PM, Thompson, John wrote:
>
> Hi Matthew,
>
> Could you point me to more information on how you got the plug-in 
> mechanism to work with clang on Windows?
>
> I understand the need to add access functions for static data, but I 
> don't understand how you build the plugin DLL in general.
>
> How does a clang.exe built from statically-linked libraries have the 
> necessary exports to satisfy the dynamic loading of the plugin DLL?
>
We made the following change to clang's tools/driver/CMakeLists.txt

    diff --git a/tools/driver/CMakeLists.txt b/tools/driver/CMakeLists.txt
    index 2545610..a3fe292 100644
    --- a/tools/driver/CMakeLists.txt
    +++ b/tools/driver/CMakeLists.txt
    @@ -10,10 +10,17 @@ set( LLVM_LINK_COMPONENTS
        selectiondag
        )

    +if(MSVC)
    +  set(def_file "clang.def")
    +else(MSVC)
    +  set(def_file "")
    +endif(MSVC)
    +
      add_clang_executable(clang
        driver.cpp
        cc1_main.cpp
        cc1as_main.cpp
    +  ${def_file}
        )

      target_link_libraries(clang
    @@ -40,6 +47,10 @@ target_link_libraries(clang

      set_target_properties(clang PROPERTIES VERSION
    ${CLANG_EXECUTABLE_VERSION})

    +if(MSVC)
    +  set_target_properties(clang PROPERTIES ENABLE_EXPORTS true)
    +endif(MSVC)
    +
      add_dependencies(clang clang-headers)

      if(UNIX)


Note that on Windows we only support CMake builds.

Documentation for ENABLE_EXPORTS is here: 
http://www.cmake.org/cmake/help/v2.8.10/cmake.html#prop_tgt:ENABLE_EXPORTS

On the plug-in side we just added 'clang' to the list of Polly's target 
link libraries.


> How do you get the list of symbols to import in the plugin DLL build?  
> Do you manually collect it from a bare link of the plug-in .obj's with 
> no library references?
>
I wish I could say I did something intelligent here, but unfortunately 
this was a manual process. I would attempt to link, note any unresolved 
externals, add them to the def file, and then relink. I don't consider 
this to be a maintainable solution in general. However at the time we 
were under pressure to release and discovered our Windows issues very 
late. So I did not spend much time thinking about a better solution to 
managing the exports.

> Or did you succeed in building a DLL-based version of clang? (Not one 
> big clang DLL like I was experimenting with, but the normal libraries 
> as DLLs.)  If so, how did you do the exports and imports in general?
>
No. We're just building a stand-alone clang executable.

> Doing so without declspec(import/export) seemed problematic because of 
> the 16K limit on the number of exports or some-such.  I know exporting 
> everything in one big DLL would exceed that, but I'm not sure about 
> separate DLLs.
>
Hmm. We did not run into any size limitations. Perhaps because we were 
exporting just what Polly needed.

> How do you register the plugin's action?  In my hack, I had to have 
> the plugin DLL provide a named function derived from the plug-in name 
> in order for clang to find it.
>
We didn't have to do anything special here. Static initializers in the 
Polly dll run on load which register the Polly passes (see 
lib/RegisterPasses.cpp in the Polly source). Is there a difference in 
the way that back-end plug-ins work versus clang AST plug-ins?

Hope this helps.

Cheers,
Matthew


> Thanks.
>
> -John
>
> *From:*Gao, Yunzhong
> *Sent:* Wednesday, March 20, 2013 2:42 PM
> *To:* Thompson, John
> *Cc:* 'Matthew Curtis'; 'cfe-commits at cs.uiuc.edu'
> *Subject:* RE: [PATCH] Implement a sane plugin API for clang
>
> Hi John,
>
> The other day I ran across a post by Matthew of the Polly team and it 
> seems that the Polly
>
> team might have made some progress regarding plug-in support on Windows.
>
> http://lists.cs.uiuc.edu/pipermail/llvm-commits/Week-of-Mon-20130311/168147.html
>
> Although I do not fully understand their implementation, it sounds 
> like the Polly team was
>
> not trying to add __declspec(dllimport/dllexport) all over the 
> clang/llvm source base. They
>
> used a .def file to export the necessary symbols from the clang 
> executable, and then they
>
> just tried to use the imported symbols without __declspec(dllimport). 
> According to the
>
> MSDN blog entry below, dllimport is not necessary for function calls 
> and is only necessary
>
> for data access. I am cc'ing Matthew in case I misunderstood Polly's 
> implementation.
>
> http://blogs.msdn.com/b/russellk/archive/2005/03/20/399465.aspx?wa=wsignin1.0
>
> I think both the Cygwin and the mingw environments on Windows provide 
> a dlltool
>
> executable that can automate the generation of .def files from either 
> the object files or a
>
> static library. I am not aware of equivalent tools in the Visual 
> Studio package; you may have
>
> to write some scripts to extract and package the symbols if you are 
> using Visual Studio.
>
> HTH,
>
> - Gao
>
> *From:*cfe-commits-bounces at cs.uiuc.edu 
> <mailto:cfe-commits-bounces at cs.uiuc.edu> 
> [mailto:cfe-commits-bounces at cs.uiuc.edu] *On Behalf Of *Thompson, John
> *Sent:* Friday, March 15, 2013 6:04 PM
> *To:* cfe-commits at cs.uiuc.edu <mailto:cfe-commits at cs.uiuc.edu>
> *Subject:* RE: [PATCH] Implement a sane plugin API for clang
>
> Hi Sean,
>
> I missed the discussion, but I saw your message and wanted to find out 
> where things stood on the plug-in API with respect to Windows.
>
> A year or two ago I wrote a plug-in for Clang for generating LUA 
> bindings, and developing on Windows, found that the plug-in mechanism 
> didn't work on Windows, because it relies on how *nix systems do 
> DLLs.  Unfortunately, in Windows land, a plug-in DLL would have to be 
> linked with Clang libraries which are DLLs, and apparently building 
> the Clang libraries as DLLs is problematic, due to the build system 
> and the messy business of importing/exporting symbols.
>
> I managed to skirt the issue by still building the plug-in against the 
> static libraries, and hacking clang to look up call an entry point 
> function with a name based on the plug-in name so it could hook up the 
> objects in the plug-in.  This, of course, was kind of dangerous, 
> because the plugin module and the clang executable would have totally 
> separate static data areas, not to mention the duplicated code segments.
>
> Then I did some work on making a big Clang DLL, to skirt some of the 
> DLL import/export issues, just exporting the symbol references needed 
> between the driver and the library, using some header conventions for 
> handling the exports.  I intended to pursue this further, and try to 
> resolve the clang DLL build issues, but haven't been able to get back 
> to it.
>
> Has anyone else worked on resolving the plugin issues for Windows or 
> the Clang-libraries-as-DLLs build?
>
> -John
>


-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130321/b834ae7e/attachment.html>