[clang] [OffloadBundler] Compress bundles over 4GB (PR #122307)

[Yaxun Liu via cfe-commits]

yxsamliu wrote:

> > > Seems fine. What's required to move HIP over to using the binary format natively by the way? Guessing we'd need to update a bunch of tools in the fork.
> > 
> > 
> > need to teach comgr to load the new offload binary
> 
> Should be easy enough, since there's a library function for that in LLVM. Is comgr the only location that would need to change? Nothing in ROCr or HIP runtime?

currently HIP runtime loads fat binary by itself for non-compressed fat binary. It needs to switch to use comgr for loading fat binary. Some math libs compile assembly code to code objects then bundle them to fat binary. Such math libs need to switch to use tools for packing offload binary. Also, zstd compression of code objects for different GPU arch by taking advantage of redundancy in ISA is critical for math libraries. The offload binary needs to support zstd compression.

https://github.com/llvm/llvm-project/pull/122307