[LLVMdev] RFC: ThinLTO Impementation Plan

[Duncan P. N. Exon Smith]

> On 2015-May-15, at 07:30, Teresa Johnson <tejohnson at google.com> wrote:
> 
>>> a. Lazy Debug Metadata Linking:
>>> 
>>> The prototype implementation included lazy importing of module-level
>>> metadata during the ThinLTO pass finalization (i.e. after all function
>>> importing is complete). This actually applies to all module-level
>>> metadata, not just debug, although it is the largest. This can be
>>> added as a separate set of patches. Changes to BitcodeReader,
>>> ValueMapper, ModuleLinker
>> 
>> It sounds like this would work well with the "full" LTO implemented
>> by tools/gold-plugin right now.  What exactly did you do to improve
>> this?
> 
> I don't think it will help with full LTO. The parsing of the metadata
> is only delayed until the ThinLTO pass finalization, and the delayed
> metadata import is necessary to avoid reading and linking in the
> metadata multiple times (for each function imported from that module).
> Coming out of the ThinLTO pass you still have all the metadata
> necessary for each function that was imported. For a full LTO that
> would end up being all of the metadata in the module.
> 
> The high level summary is that during the initial import it leaves the
> temporary metadata on the instructions that were imported, but saves
> the index used by the bitcode reader used to correlate with the
> metadata when it is ready (i.e. the MDValuePtrs index), and skips the
> metadata parsing. During finalization we parse just the metadata, and
> suture it up during metadata value mapping using the saved index.

AFAICT, the gold-plugin currently does similar work.  (Rafael knows
this code better (I've only introduced bugs there), but IIRC he's on
vacation until next week.)  Even in "full" LTO.

Have a look at `getModuleForFile()` and its calling loop inside
`allSymbolsReadHook()`:

 1. Load a single module, lazily.
 2. Delete the bodies of unwanted functions (without ever loading them)
    and fiddle with linkage as necessary.
 3. Link in the module.
 4. Delete the module.

How is what you're proposing different?  Does it need to be different?