[PATCH] D64461: [lld-link] implement -thinlto-index-only

[Teresa Johnson via Phabricator via llvm-commits]

tejohnson added a comment.

In D64461#1586582 <https://reviews.llvm.org/D64461#1586582>, @rnk wrote:

> In D64461#1586526 <https://reviews.llvm.org/D64461#1586526>, @smeenai wrote:
>
> > Distributed ThinLTO doesn't support archives: http://lists.llvm.org/pipermail/llvm-dev/2019-June/133145.html. @tejohnson suggested `-start-lib -end-lib` as a workaround, but that's not a thing for COFF. Do you have plans for archive support, or is that not something you need to deal with for your use cases?
>
>
> Personally I believe we should prioritize fixing that. It's trivial to fix for thin archives, since those are practically equivalent to the command line object groups.


Someone asked about this on the llvm-dev mailing list recently, pasting my response below:

Somehow, the module identifier for each constituent object would need to be both unique (we currently generate this from the name of the archive plus the offset in the archive plus the name of the source file IIRC), but also correctly identify the extracted bitcode object used in the post-thinlink backend invocation. This is so it can write out the distributed index file with a filename that gets consumed by the associated backend invocation (passed to -fthinlto-index=), and so that the module paths emitted in those index files correctly identify where we can import functions from. Since the bitcode objects need to be extracted for the corresponding backend clang invocations, a couple possibilities come to mind:

1. Do it outside the compiler/linker: wrap the whole thing in a script that does the extraction, invokes the link with extracted constituents surrounded by --start-lib/-end-lib pairs, invokes each backend through some parallel or distributed mechanism, and then invokes the final link; or
2. Add support to pass some kind of mapping file into LTO that maps from each archive constituent to the extracted filename including path that the corresponding ThinLTO backend clang invocation will use, and have LTO set the module identifiers accordingly so that everything "just works" (in theory). We already support some munging of these names (see the thinlto_object_suffix_replace plugin option in either gold-plugin.cpp or in lld), but what you need here is a bit more complicated than a simple suffix change. But since there is already support for adjusting the name, it might not be too bad to add this support.


Repository:
  rL LLVM

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D64461/new/

https://reviews.llvm.org/D64461