[PATCH] D130229: [ELF] Add --thinlto-index= and --remapping-file=

[Teresa Johnson via Phabricator via llvm-commits]

tejohnson added a comment.

In D130229#3669613 <https://reviews.llvm.org/D130229#3669613>, @MaskRay wrote:

> In D130229#3668618 <https://reviews.llvm.org/D130229#3668618>, @tejohnson wrote:
>
>> I suppose the rationale for doing things this way is that you can keep the input files in the same ordering on the indexing link and the final native link, to avoid changing the relative order between files that were bitcode and files that were native already during the indexing link.
>
> Yes, and also to avoid `-nostdlib`.
>
>> However, I'm a little confused as to how this will work for symbol resolution. The main reason for doing things the way we were was to ensure the same objects were selected for linking out of static libraries (that use --start-lib/--end-lib), since symbols may be referenced differently after importing. That's why the native objects corresponding to the bitcode files are listed in a flat list in the thinlto-index-only=params file (i.e. not using --start-lib/--end-lib). I'm not sure how the new solution prevents those issues.
>>
>> E.g. If I change your example to use --start-lib/--end-lib:
>>
>>   # ThinLTO indexing
>>   clang -flto=thin -fuse-ld=lld -Wl,--thinlto-index=out/exe.map -Wl,--thinlto-prefix-replace='out;lto' \
>>     -Wl,--thinlto-object-suffix-replace='.indexing.o;.o' out/a.o --start-lib out/b.indexing.o out/c.indexing.o --end-lib
>>
>> Is the final link also like that?:
>
> Yes for all input files. The final link removes `-flto=thin -Wl,--thinlto-index=out/exe.map` and adds `-Wl,--remapping-file=out/exe.map`
>
> `--thinlto-index-only=` reorders bitcode files and native object files, therefore may change symbol resolution. As a compensation, all backend compiled bitcode files drop --start-lib/--end-lib.
> For `--thinlto-index=`, there is no reordering. I am trying to think of a case where dropping `--start-lib` is a necessity but so far unable to come up with one.

See discussion and examples linked from https://reviews.llvm.org/D22677

> If there is a case, we probably have to give --remapping-file additional semantics that a specified file ignores the surrounding `--start-lib`. This makes  --remapping-file less generic, but we can probably add the third column to indicate whether --start-lib should be ignored.

Can we just have the file generated by thinlto-index-only= also include native objects?

> The complexity stems from the chain reaction of archive member extraction (I use the term for lazy object files as well), which is not easily formulated formerly.
>
>>   # Final link
>>   clang -fuse-ld=lld -Wl,--remapping-file=out/exe.map out/a.o --start-lib out/b.indexing.o out/c.indexing.o --end-lib -o out/exe
>>
>> I assume the symbol resolution is done on the post-mapping native objects, and not the bitcode objects, right? In more complex examples that could cause issues with which objects are selected, leading to similar problems as those this is trying to address relating to originally native objects.
>
> Yes.




Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D130229/new/

https://reviews.llvm.org/D130229