[PATCH] D15390: [ThinLTO] Launch importing backends in parallel threads from gold plugin

[Teresa Johnson via llvm-commits]

tejohnson created this revision.
tejohnson added reviewers: joker.eph, rafael, dexonsmith.
tejohnson added subscribers: llvm-commits, davidxl.
Herald added a subscriber: joker.eph.

This is still WIP in that it hasn't been well-tested yet. Sending this
for early comments. It works for a small example.

Instead of exiting after creating the combined index in the gold plugin,
unless requested by new option, we will now launch the ThinLTO backends
(LTO and codegen pipelines with importing) in parallel threads. The
number of threads is controlled by the existing -jobs gold plugin option,
or the hardware concurrency if not specified.

There are a few things here that I plan to commit separately soon:
- changing the FunctionIndex pointer on the PassManagerBuilder to a
  const pointer.
- invoking release_input_file after merging in each module's index into
  the combined index (bug exposed now that we aren't exiting right
  after)
- Some clang-format cleanup of header file orders.

Also, for now the thread management is pretty simple: if we hit the max
number of threads launched, we wait for the oldest to complete before
launching the next. This would be better with a thread pool, but I don't
see any existing support in LLVM, so I presumably need to roll my own.

http://reviews.llvm.org/D15390

Files:
  include/llvm/Transforms/IPO.h
  include/llvm/Transforms/IPO/PassManagerBuilder.h
  lib/Transforms/IPO/FunctionImport.cpp
  tools/gold/gold-plugin.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D15390.42323.patch
Type: text/x-patch
Size: 11836 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20151209/d6673a55/attachment.bin>