[PATCH] D125165: [Clang] Introduce clang-offload-packager tool to bundle device files

[Yaxun Liu via Phabricator via cfe-commits]

yaxunl added a comment.

In D125165#3506529 <https://reviews.llvm.org/D125165#3506529>, @jhuber6 wrote:

> In D125165#3506502 <https://reviews.llvm.org/D125165#3506502>, @thakis wrote:
>
>> In D125165#3506477 <https://reviews.llvm.org/D125165#3506477>, @yaxunl wrote:
>>
>>> In D125165#3506448 <https://reviews.llvm.org/D125165#3506448>, @thakis wrote:
>>>
>>>> We could add a "clang-offload-bundler and clang-offload-wrapper are deprecated, replace them with $whatever" in the release notes and then remove them a release later, assuming the replacement is straightforward.
>>>
>>> I think it is still too early to say clang-offload-bundler is deprecated. It is used by HIP toolchain and has functionality currently not available in clang-offload-packager.
>>
>> If I read the above right, jhuber says it's been merged into clang itself, not that it's being replaced by clang-offload-packager (?)
>
> I'll clarify, the functionality of the `clang-offload-bundler` is to embed device files into the host. I now do this directly in clang by creating a global string in the LLVM-IR of the host rather than calling a tool. The HIP toolchain still uses the `clang-offload-bundler`, but I'm planning on putting patches up to move away from that. The current `clang-offload-bundler` and this new tool have different purposes, this one simply create a binary that can then be embedded into the host. There is still functionality that the `clang-offload-bundler` provides that I don't intend to replace, namely the bundling and un-bundling of text files. I don't think we want to stick with the `clang-offload-bundler` approach, because the files that the `--clang-offload-bundler` spat out weren't valid input to the rest of LLVM, e.g. `clang -S -emit-llvm --offload-arch=gfx908 foo.hip -o - | opt` would break.

The clang-offload-bundler textual format can be consumed by clang since clang can automatically unbundle them.

The textural format allows clang to emit one output for -E, which can be used by ccache for calculating hashes.

Another usage of clang-offload-bundler textual format is bundled assembly code. Users can modify them and use clang to assemble them.

For embedding bitcode in relocatable object files, clang-offload-packager can be a replacement for clang-offload-bundler, since this is consumed by compiler.

For HIP toolchain, clang-offload-bundler is also used to generate fatbinary files which can be loaded dynamically at run time through module API's. So far I don't think this can be replaced by clang-offload-packager in a short time, since it needs HIP runtime change.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D125165/new/

https://reviews.llvm.org/D125165