[all-commits] [llvm/llvm-project] af382e: [LinkerWrapper] Support relocatable linking for of...

[Joseph Huber via All-commits]

  Branch: refs/reviewable/pr80066/r4
  Home:   https://github.com/llvm/llvm-project
  Commit: af382e03e41ef679c35a6126a1b131a7a8a28360
      https://github.com/llvm/llvm-project/commit/af382e03e41ef679c35a6126a1b131a7a8a28360
  Author: Joseph Huber <huberjn at outlook.com>
  Date:   2024-01-31 (Wed, 31 Jan 2024)

  Changed paths:
    M clang/test/Driver/linker-wrapper-image.c
    M clang/test/Driver/linker-wrapper.c
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp
    M llvm/include/llvm/Frontend/Offloading/OffloadWrapper.h
    M llvm/lib/Frontend/Offloading/OffloadWrapper.cpp
    M llvm/lib/Object/OffloadBinary.cpp

  Log Message:
  -----------
  [LinkerWrapper] Support relocatable linking for offloading

Summary:
The standard GPU compilation process embeds each intermediate object
file into the host file at the `.llvm.offloading` section so it can be
linked later. We also use a sepcial section called something like
`omp_offloading_entries` to store all the globals that need to be
registered by the runtime. The linker-wrapper's job is to link the
embedded device code stored at this section and then emit code to
register the linked image and the kernels and globals in the offloading
entry section.

One downside to RDC linking is that it can become quite big for very
large projects that wish to make use of static linking. This patch
changes the support for relocatable linking via `-r` to support a kind
of "partial" RDC compilation for offloading languages.

This primarily requires manually editing the embedded data in the
output object file for the relocatable link. We need to rename the
output section to make it distinct from the input sections that will be
merged. We then delete the old embedded object code so it won't be
linked further. We then need to rename the old offloading section so
that it is private to the module. A runtime solution could also be done
to defer entires that don't belong to the given GPU executable, but this
is easier. Note that this does not work with COFF linking, only the ELF
method for handling offloading entries, that could be made to work
similarly.

Given this support, the following compilation path should produce two
distinct images for OpenMP offloading.
```
$ clang foo.c -fopenmp --offload-arch=native -c
$ clang foo.c -lomptarget.devicertl --offload-link -r -o merged.o
$ clang main.c merged.o -fopenmp --offload-arch=native
$ ./a.out
```

Or similarly for HIP to effectively perform non-RDC mode compilation for
a subset of files.

```
$ clang -x hip foo.c --offload-arch=native --offload-new-driver -fgpu-rdc -c
$ clang -x hip foo.c -lomptarget.devicertl --offload-link -r -o merged.o
$ clang -x hip main.c merged.o --offload-arch=native --offload-new-driver -fgpu-rdc
$ ./a.out
```

One question is whether or not this should be the default behaviour of
`-r` when run through the linker-wrapper or a special option. Standard
`-r` behavior is still possible if used without invoking the
linker-wrapper and it guranteed to be correct.


  Commit: 1a91a8a1a5bd0a0b6d47dd0e50e801a820400203
      https://github.com/llvm/llvm-project/commit/1a91a8a1a5bd0a0b6d47dd0e50e801a820400203
  Author: Joseph Huber <huberjn at outlook.com>
  Date:   2024-01-31 (Wed, 31 Jan 2024)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp
    M llvm/include/llvm/Frontend/Offloading/OffloadWrapper.h

  Log Message:
  -----------
  Address comments


  Commit: 7db493af1506f93e5a8ba02183194633746f5d93
      https://github.com/llvm/llvm-project/commit/7db493af1506f93e5a8ba02183194633746f5d93
  Author: Joseph Huber <huberjn at outlook.com>
  Date:   2024-01-31 (Wed, 31 Jan 2024)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  Format


  Commit: 4c480e8a242689371744ee56d1a7d51323b50460
      https://github.com/llvm/llvm-project/commit/4c480e8a242689371744ee56d1a7d51323b50460
  Author: Joseph Huber <huberjn at outlook.com>
  Date:   2024-01-31 (Wed, 31 Jan 2024)

  Changed paths:
    M clang/docs/ClangLinkerWrapper.rst
    M clang/docs/OffloadingDesign.rst
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp
    M clang/tools/clang-linker-wrapper/LinkerWrapperOpts.td

  Log Message:
  -----------
  Add Docs


  Commit: 8c90e6dd88b1d556dfc99ccfc3c93e4ea8f7d8d1
      https://github.com/llvm/llvm-project/commit/8c90e6dd88b1d556dfc99ccfc3c93e4ea8f7d8d1
  Author: Joseph Huber <huberjn at outlook.com>
  Date:   2024-01-31 (Wed, 31 Jan 2024)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  Format


Compare: https://github.com/llvm/llvm-project/compare/af382e03e41e%5E...8c90e6dd88b1