[llvm-branch-commits] [clang] [llvm] [OffloadBundler] Bound compressed bundles by header size, not magic scan (PR #206745)

[via llvm-branch-commits]

michaelselehov wrote:

When several offload bundles are concatenated in a single section/file, the
unbundler (`clang-offload-bundler`) and `llvm-objdump --offloading` need to find
where one bundle ends and the next begins. For compressed bundles they did this
by scanning for the next occurrence of the `"CCOB"` magic string starting right
after the current header:

```cpp
NextBundleStart = Buffer.find("CCOB", 4);
...take_front(NextBundleStart)...
```

A zstd/zlib-compressed payload can legally contain the four bytes `"CCOB"`
anywhere inside it. When that happens, the scan stops in the middle of the
compressed data and the bundle is truncated, which corrupts the embedded code
object. In practice this surfaced as a "decomposition" failure for hipBLASLt
bf16 GEMMs on gfx942: the real bundle is 802104 bytes, but its payload contains
`"CCOB"` at offset 585084, so the unbundler handed a 585084-byte truncated blob
to the decompressor.

Fix:

- The compressed bundle header already records the authoritative total size of
  the bundle (`FileSize`, present in format V2/V3). Use it to compute the exact
  bundle boundary, and only scan for the next magic *past* that point.
- Bundles without a recorded size (legacy V1) keep the previous magic-scan
  behavior as a fallback.

This is applied to all three scanning sites: `OffloadBundler::ListBundleIDsInFile`
and `OffloadBundler::UnbundleFiles` in the Clang driver, and
`extractOffloadBundle` in `llvm::object`. A small helper
`getCompressedBundleSize` parses the header and returns `FileSize`.

Tests:

- A skippable-frame fixture that deliberately embeds `"CCOB"` inside the
  compressed payload exercises the boundary logic from both
  `clang-offload-bundler` (`clang/test/Driver/clang-offload-bundler-magic-collision.c`)
  and `llvm-objdump --offloading`
  (`llvm/test/tools/llvm-objdump/Offloading/fatbin-magic-collision.test`).

History / why this is safe now:

- An earlier version of this fix (#205587) was reverted because it relied on the
  `FileSize` field while the header was still read in host-native byte order,
  which broke big-endian targets. This version sits on top of the little-endian
  header fix, so `FileSize` is read correctly on every host.

Validation:

- x86_64: full `check-llvm` / `check-clang` and the new tests pass.
- s390x (big-endian, qemu-user): on the real fixture the new FileSize boundary
  keeps the bundle whole (802104), whereas the old magic-scan boundary truncates
  it at 585084.

Assisted-by: Claude Opus


https://github.com/llvm/llvm-project/pull/206745