[PATCH] D23680: [ThinLTO] Emit files for distributed builds for all modules

[Teresa Johnson via llvm-commits]

tejohnson created this revision.
tejohnson added a reviewer: mehdi_amini.
tejohnson added subscribers: pcc, llvm-commits.
Herald added a subscriber: mehdi_amini.

With the new LTO API in r278338, we stopped emitting the individual
index files and imports files for some modules in the distributed backend
case (thinlto-index-only plugin option).

They were no longer being emitted for files that didn't contain a
module index (e.g. when they had inline assembly, since we can't currently
safely import), because the new API only added them to the ThinLTO backend
based on the presence of a module index. They were instead added as
regular LTO files, and were all expected to be merged into a single LTO
module. This is problematic because the distributed build system will
not know how to create backend actions for those files. Additionally,
the lack of generated individual index and import files makes error
checking the existence of expected outputs of the thin link step very
difficult, and those files will not be available for the subsequent
ThinLTO backend process. To address this particular issue, add a new LTO
Config flag to force ThinLTO mode, set it when thinlto-index-only
specified, and use it to add all modules to the ThinLTO backend.

Another case is when the linker decides not to include a module in the
link, because it was in an archive library and did not have a strong
reference to it. Again, not creating the expected output files makes the
distributed build system implementation more difficult, in terms of
checking for the expected outputs of the thin link, and scheduling the
backend jobs. To address this, added an interface for specifying the
module paths of such non linked modules, and added handling to emit the
expected output files in the WriteIndexesThinBackend.

Added tests for both of these cases. For the former, I added the
thinlto-index-only option to llvm-lto2 which will result in creation of
the WriteIndexesThinBackend. I therefore have tests for both gold and
llvm-lto2. For the latter problem, augmented a gold v1.12+ test, since
that version of gold has the handling for notifying on modules not being
included in the link.

https://reviews.llvm.org/D23680

Files:
  include/llvm/LTO/Config.h
  include/llvm/LTO/LTO.h
  lib/LTO/LTO.cpp
  test/ThinLTO/X86/Inputs/emit_imports.ll
  test/ThinLTO/X86/Inputs/empty.ll
  test/ThinLTO/X86/emit_imports.ll
  test/tools/gold/X86/Inputs/thinlto_empty.ll
  test/tools/gold/X86/thinlto_emit_imports.ll
  test/tools/gold/X86/v1.12/thinlto_emit_linked_objects.ll
  tools/gold/gold-plugin.cpp
  tools/llvm-lto2/llvm-lto2.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D23680.68572.patch
Type: text/x-patch
Size: 14196 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160818/4a86b977/attachment-0001.bin>