[llvm-dev] JIT, LTO and @llvm.global_ctors: Looking for advise

[Benoit Belley via llvm-dev]

Hi Everyone,

We are looking for advise regarding the proper use of LTO in
conjunction with just-in time generated code. Our usage scenario goes
as follows.

  1. Our front-end generates an LLVM module.

  2. A small runtime support library is linked-in. The runtime
     library is distributed as bitcode. It is generated using "clang++
     -emit-llvm' and 'llvm-link'. This allows LTO to kick-in and
     functions from the runtime support library become candidates for
     inlining. This is the desired effect that we are looking for.

  3. The resulting LLVM module is compiled and dynamically loaded. We
     are currently using the MCJIT API, but are planning to move to ORC
     very soon.

Our LLVM module linking code goes roughly as follows:

  Linker linker(jittedModule);
  std::unique_ptr<llvm::Module> moduleToLink(
    getLazyIRFileModule(bcFileName, error, context));
  linker.linkInModule(std::move(module),
                      Linker::LinkOnlyNeeded |
                      Linker::InternalizeLinkedSymbol);

Our issue is with the Linker::LinkOnlyNeeded flag. Using it has a huge
positive impact on link and compilation time :-). But, it causes the
@llvm.global_ctors and @llvm.global_dtors references from the
linked-in modules to be discarded :-(. AFAICT, the Linker code assumes
ThinLTO when the LinkOnlyNeeded flags is specified, and full-LTO
otherwise.

To resolve this, we have locally patched
llvm/lib/Linker/LinkModules.cpp with:

  bool ModuleLinker::run() {

    // ....

    if (shouldImportIntrinsicGlobalVariables()) {
      auto addIntrinsicGlobalVariable = [ValuesToLink,
                                         srcM](llvm::StringRef name) {
        if (GlobalValue *GV = SrcM->getGlobalVariable(name)) {
          ValuesToLink.insert(GV);
        }
      };

      // These are added here, because they must not be internalized.
      addIntrinsicGlobalVariable("llvm.used");
      addIntrinsicGlobalVariable("llvm.compiler.used");
      addIntrinsicGlobalVariable("llvm.global_ctors");
      addIntrinsicGlobalVariable("llvm.global_dtors");
    }

    // ...

  }

Questions:

   1. Is attempting to use llvm::Linker appropriate for our usage
      pattern ? Should we directly use llvm::IRMover instead ?

   2. Or, is our patch to ModuleLinker::run() the way to go ? Should
      we submit back a patch along these lines ?

   3. Other suggestions ?

[Note] We are currently using LLVM 4.0.1-rc2.

Cheers,
Benoit



Benoit Belley
Sr Principal Developer
M&E-Product Development Group
 
MAIN +1 514 393 1616
DIRECT +1 438 448 6304
FAX +1 514 393 0110
 
Twitter <http://twitter.com/autodesk>
Facebook <https://www.facebook.com/Autodesk>
 
Autodesk, Inc.
10 Duke Street
Montreal, Quebec, Canada H3C 2L7
www.autodesk.com <http://www.autodesk.com/>