[PATCH] D20268: [wip] Resolution-based LTO API.

[Mehdi AMINI via llvm-commits]

joker.eph added a comment.

Haven't had much time to look at anything in the implementation yet, as what seems important to me is the layering and separation of component. I'd expect anything the linker needs to setup / interact with to be concentrate in some place (lib/LTO or equivalent). What I'd try to avoid is exposing LLVM internals that are not relevant to the linker (like the LLVMContext for instance).

Below is a quick brain-dump of a high-level view of how I'd expect the various layers of the LTO machinery to be laid-down:

- A common sequential "thin" phase that performs IPA over the summary, and compute all the information needed by each backend task.
- The information for each ThinLTO "backend-task" are packaged and forwarded to a custom LTOBackend implementation (which component/level handles the threading is fuzzy for now, but both scheme could be threaded I think):
  - if in-memory, each task take the result of the IPA and setup a `ThinLTOProcessor` (strawman name I just made up) that actually load the IR, performs the import, codegen, etc.
  - if "distributed", each task that takes the result of the IPA and serialize it in a "thinlto-package" for the distributed build system.

In the "distributed" mode, I assume the linker stops here and yield to the build system, which distributes the "thinlto-package". The distributed job run clang, which unserialize the "thinlto-package" and setup a `ThinLTOProcessor` the same way the in-memory backend does.

Hopefully the part that setup the pipeline/targetmachine/codegen/... will be common between regular LTO and ThinLTO individual tasks.
Not much of this needs to be exposed to the linker though. Let's keep the API exposed to the linker as small as possible. 
Some parts are fairly "easy":

- "Load this LTO memory buffer" (returns an `LTOObjectFile`), could be a free function? Along these lines: `std::unique_ptr<LTOObjectFile> createLTOObjectFile(MemBufferRef Buffer)`. This can be lazy (i.e. don't do anything but verifying the bitcode signature).
- "Iterate through the symbols for this `LTOObjectFile`: maybe a `LTOSymbol` class that exposes whatever level of information we would be exposing "per symbol". We can next think about laying out these informations in the bitcode as a symbol table that would be more efficient (i.e, not requiring much parsing/loading in memory). There's a PR that Rafael opened for that. Ultimately we shouldn't need to load a Module and setup a LLVMContext to get the symbol names, their linkage type, their section, etc. So I wouldn't care about optimizing the API to handle the LLVMContext in particular.
- Set the `SymbolResolution` for every `LTOSymbol`.

This is somehow covered in this patch.

What does not seem nailed down yet (or is not clear to me at least, maybe I need to start looking closer at the implementation) is how to setup a flavor of LTOBackend: with/without distributed build for instance, providing the appropriate configuration, diagnostic handler, and hooks (for adding a stream and such).

(As a general comment: free functions are convenient (no hidden state) but they'd require to pass a diagnostic handler all the time which is annoying)


http://reviews.llvm.org/D20268