[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

[David Blaikie via llvm-dev]

+Lang (for JIT) & Teresa (for LTO/ThinLTO).

Sounds like maybe the LinkOnlyNeeded got reused for a bit more than the
original intent & maybe there should be more than one flag here - not sure.

On Mon, Jun 19, 2017 at 9:16 AM Benoit Belley via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Hi Everyone,
>
> We are looking for advise regarding the proper use of LTO in
> conjunction with just-in time generated code. Our usage scenario goes
> as follows.
>
>   1. Our front-end generates an LLVM module.
>
>   2. A small runtime support library is linked-in. The runtime
>      library is distributed as bitcode. It is generated using "clang++
>      -emit-llvm' and 'llvm-link'. This allows LTO to kick-in and
>      functions from the runtime support library become candidates for
>      inlining. This is the desired effect that we are looking for.
>
>   3. The resulting LLVM module is compiled and dynamically loaded. We
>      are currently using the MCJIT API, but are planning to move to ORC
>      very soon.
>
> Our LLVM module linking code goes roughly as follows:
>
>   Linker linker(jittedModule);
>   std::unique_ptr<llvm::Module> moduleToLink(
>     getLazyIRFileModule(bcFileName, error, context));
>   linker.linkInModule(std::move(module),
>                       Linker::LinkOnlyNeeded |
>                       Linker::InternalizeLinkedSymbol);
>
> Our issue is with the Linker::LinkOnlyNeeded flag. Using it has a huge
> positive impact on link and compilation time :-). But, it causes the
> @llvm.global_ctors and @llvm.global_dtors references from the
> linked-in modules to be discarded :-(. AFAICT, the Linker code assumes
> ThinLTO when the LinkOnlyNeeded flags is specified, and full-LTO
> otherwise.
>
> To resolve this, we have locally patched
> llvm/lib/Linker/LinkModules.cpp with:
>
>   bool ModuleLinker::run() {
>
>     // ....
>
>     if (shouldImportIntrinsicGlobalVariables()) {
>       auto addIntrinsicGlobalVariable = [ValuesToLink,
>                                          srcM](llvm::StringRef name) {
>         if (GlobalValue *GV = SrcM->getGlobalVariable(name)) {
>           ValuesToLink.insert(GV);
>         }
>       };
>
>       // These are added here, because they must not be internalized.
>       addIntrinsicGlobalVariable("llvm.used");
>       addIntrinsicGlobalVariable("llvm.compiler.used");
>       addIntrinsicGlobalVariable("llvm.global_ctors");
>       addIntrinsicGlobalVariable("llvm.global_dtors");
>     }
>
>     // ...
>
>   }
>
> Questions:
>
>    1. Is attempting to use llvm::Linker appropriate for our usage
>       pattern ? Should we directly use llvm::IRMover instead ?
>
>    2. Or, is our patch to ModuleLinker::run() the way to go ? Should
>       we submit back a patch along these lines ?
>
>    3. Other suggestions ?
>
> [Note] We are currently using LLVM 4.0.1-rc2.
>
> Cheers,
> Benoit
>
>
>
> Benoit Belley
> Sr Principal Developer
> M&E-Product Development Group
>
> MAIN +1 514 393 1616 <(514)%20393-1616>
> DIRECT +1 438 448 6304 <(438)%20448-6304>
> FAX +1 514 393 0110 <(514)%20393-0110>
>
> Twitter <http://twitter.com/autodesk>
> Facebook <https://www.facebook.com/Autodesk>
>
> Autodesk, Inc.
> 10 Duke Street
> Montreal, Quebec, Canada H3C 2L7
> www.autodesk.com <http://www.autodesk.com/>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170620/8bc3ab44/attachment.html>