[LLVMdev] How to do bitcode archive linking correctly?

[Daniel Liew]

Hi,

Daniel & Eli. Would you be able to comment on the right approach to the 
problem discussed below seeing as it appears you both worked on the 
commits that removed the ability to link in bitcode archives?

I'd be especially keen to here your opinion Daniel as you are the 
original author of KLEE so you probably have a good idea what would be a 
suitable replacement (invoking gold with the LLVM LTO plug-in via exec() 
does not sound very appealing).


On 17/12/13 10:31, Daniel Liew wrote:
> Hi,
>
> We're currently upgrading KLEE to work with LLVM >=3.3 and we've hit a problem.
>
> It seems r172749 removed support for linking a bitcode archive into a
> module. KLEE unfortunately depends on this to link in its runtime (
> which amongst other things provides a C library [5] ).
>
> A first attempt at linking in a bitcode archive ourselves can be seen in [1].
>
> This approach does not work correctly, it's incredibly slow and I'm
> not convinced it will resolve all undefined or weak symbols like
> linking with an archive normally should. This leads me to ask
>
> * How does Linker::LinkModules() [2] behave? Does it just blindly
> concatenate modules ( that might explain the slow down ) or does it
> actually check the symbol tables of the modules and only link in `Src`
>   if `Dest` has unresolved or weak symbols that `Src` can provide?
>
> * If Linker::LinkModules() does decide to merge `Src` into `Dest` does
> it merge the entire contents of the `Src` module or just the symbols
> that are needed by `Dest`?
>
> * IIRC when linking archives a linker must must link constituent
> modules multiple times until a fixed point is reach (the set of
> undefined symbols does not change ). Does that apply here? I guess the
> answer depends on the behaviour of Linker:: LinkModules(). If it does
> that would make [1] incorrect as it only links in the archive's
> modules once.
>
> * Why is the approach so slow compared to using the old
> Linker::linkInFile() API in prior LLVM versions?
>
> What ever the answers are to the above, the approach in [1] isn't
> right. So what is the right way to programmatically link in LLVM
> bitcode archives into a bitcode module?
>
> I noted that the r172749 commit says...
>
> ```
> the "right" way to get this support is to use the
> platform-specific linker-integrated LTO mechanisms, or the forthcoming
> LLVM linker.
> ```
>
> I'm not sure how this helps us
>
> - For "platform-specific linker-integrated LTO mechanisms", is that
> referring to [3]? If so that's not particularly helpful as I'd like an
> API I can use to do the archive linking rather than having to invoke
> an external program (i.e. my system's linker).
> - For "forthcoming LLVM linker", is that referring to lld [4]?
>
> [1] https://github.com/MartinNowack/klee/blob/51d4a2b34511a8d6b00b16a50f90b0bc1a793a69/lib/Module/ModuleUtil.cpp#L61
> [2] http://llvm.org/docs/doxygen/html/classllvm_1_1Linker.html#a244da8c7e9789b1b675b9741bd692c63
> [3] http://llvm.org/docs/GoldPlugin.html
> [4] http://lld.llvm.org/
> [5] https://github.com/ccadar/klee-uclibc/tree/klee_0_9_29
>
> Thanks,
> Dan Liew.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>