[PATCH] D144596: Add extra parameter to thinlto-prefix-replace

[Fangrui Song via Phabricator via llvm-commits]

MaskRay added a comment.

(I'll be be out for most of the next 4 weeks and may not have much time to review patches.)

Let me write down my understanding and the proposal.

When compiling a source file with `-flto=thin`, we additionally specify `-fthin-link-bitcode=` to create a minimized bitcode file.

Here is an example (simplified from how Bazel implements distributed ThinLTO):

  for i in a b c; do clang -c -O2 -flto=thin -fthin-link-bitcode=$i.indexing.o $i.c; done
  clang -flto=thin -fuse-ld=lld -Wl,--thinlto-index-only=a.rsp -Wl,--thinlto-prefix-replace=';lto/',--thinlto-object-suffix-replace='.indexing.o;.o' elf0.o a.indexing.o -Wl,--start-lib [bc].indexing.o -Wl,--end-lib elf1.o
  clang -c -O2 -fthinlto-index=lto/a.o.thinlto.bc a.o -o lto/a.o
  clang -c -O2 -fthinlto-index=lto/b.o.thinlto.bc b.o -o lto/b.o
  clang -c -O2 -fthinlto-index=lto/c.o.thinlto.bc c.o -o lto/c.o
  clang -fuse-ld=lld @a.rsp elf0.o elf1.o  # a.rsp contains lto/a.o and lto/b.o

In the thin link, we specify `--thinlto-object-suffix-replace=.indexing.o;.o` to change IR module names.

In the emitted index files (`*.thinlink.bc`) and import files (`*.imports`), we will see `[abc].o` instead of `[abc].indexing.o`.

  % llvm-bcanalyzer --dump lto/a.o.thinlto.bc | grep '<ENTRY'
      <ENTRY abbrevid=6 op0=0 op1=97 op2=46 op3=111/> record string = 'a.o'
      <ENTRY abbrevid=6 op0=1 op1=98 op2=46 op3=111/> record string = 'b.o'

Note, thin links only require minimized bitcode files, while backend compiles only require full bitcode files.

To support tree artifacts with an unknown number of full bitcode files and minimized bitcode files, we need to use different directories.
If these bitcode files are within the same directory, Bazel will have to ship both full and minimized bitcode files to a backend compile action, causing waste.
To ship just full bitcode files in a backend compile action, full and minimized bitcode files need to be in different directories (e.g. `a.bc` and `indexing/a.bc`).

While we're here, let's make two more changes. First, let's use `.bc` instead of `.o` for the bitcode file extension name.
Second, let's place the initially compiled bitcode files in `thin/` instead of the current working directory for clarity.

  mkdir -p thin indexing
  for i in a b c; do clang -c -O2 -flto=thin -fthin-link-bitcode=indexing/$i.bc $i.c -o thin/$i.bc; done
  clang -flto=thin -fuse-ld=lld -Wl,--thinlto-index-only=a.rsp -Wl,--thinlto-prefix-replace='thin/;index/;obj/;indexing/' elf0.o indexing/a.bc -Wl,--start-lib indexing/[bc].bc -Wl,--end-lib elf1.o
  clang -c -O2 -fthinlto-index=index/a.bc.thinlto.bc a.bc -o obj/a.o  # index/a.bc.imports specifies the needed full bitcode files, e.g. thin/a.bc
  clang -c -O2 -fthinlto-index=index/b.bc.thinlto.bc b.bc -o obj/b.o
  clang -c -O2 -fthinlto-index=index/c.bc.thinlto.bc c.bc -o obj/c.o
  clang -fuse-ld=lld @a.rsp elf0.o elf1.o  # a.rsp specifies obj/a.o and obj/b.o

In `-Wl,--thinlto-prefix-replace=';index/;obj/;indexing/'`, we introduce two new components.
The third component, `obj/`, specifies the output directory for the backend compile.
For tree artifacts, an ouput directory can only be used by one action type.
Since `index/` is created by the thin link, we cannot place backend compiled ELF relocatable object files (`*.o`) there.
Backend compiled ELF relocatable object files have to use a separate directory (`obj/`).

The fourth component, `indexing/`, specifies the directory for minimized bitcode files and replaces the previous `--thinlto-object-suffix-replace='.indexing.o;.o'`.
When ld.lld communicates module names to LLVM LTO, it forms the module identifier `a.bc` from the input filename `indexing/a.bc` (strip the first component (an empty string) and prepends the fourth component `indexing/`).


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144596/new/

https://reviews.llvm.org/D144596