[PATCH] D19556: [ThinLTO] Emit individual index files for distributed backends

[Teresa Johnson via llvm-commits]

tejohnson created this revision.
tejohnson added a reviewer: joker.eph.
tejohnson added a subscriber: llvm-commits.
tejohnson added a dependency: D19481: [ThinLTO] Use valueid instead of bitcode offsets in combined index file.
Herald added a subscriber: joker.eph.

When launching ThinLTO backends in a distributed build (currently
supported in gold via the thinlto-index-only plugin option), emit
an individual index file for each backend process as described here:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

The individual index file encodes the summary and module information
required for implementing the importing/exporting decisions made
for a given module in the thin link step.
This is in place of the current mechanism that uses the combined index
to make importing decisions in each back end independently. It is an
enabler for doing global summary based optimizations in the thin link
step (which will be recorded in the individual index files), and reduces
the size of the index that must be sent to each backend process, and
the amount of work to scan it in the backends.

Rather than create entirely new ModuleSummaryIndex structures (and all
the included unique_ptrs) for each backend index file, a map is created
to record all of the GUID and summary pointers needed for a particular
index file. The IndexBitcodeWriter walks this map instead of the full
index (hiding the details of managing the appropriate summary iteration
in a new iterator subclass). This is more efficient than walking the
entire combined index and filtering out just the needed summaries during
each backend bitcode index write.

Depends on D19481.

http://reviews.llvm.org/D19556

Files:
  include/llvm/Bitcode/ReaderWriter.h
  lib/Bitcode/Writer/BitcodeWriter.cpp
  test/tools/gold/X86/thinlto.ll
  tools/gold/gold-plugin.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D19556.55076.patch
Type: text/x-patch
Size: 22704 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160426/84b4b998/attachment.bin>