[clang] [OffloadBundler] Compress bundles over 4GB (PR #122307)

Mon Jan 20 17:46:10 PST 2025

llvm-ci wrote:

LLVM Buildbot has detected a new failure on builder `llvm-clang-x86_64-gcc-ubuntu` running on `sie-linux-worker3` while building `clang` at step 6 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/174/builds/11719

<details>
<summary>Here is the relevant piece of the build log for the reference</summary>

```
Step 6 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'Clang :: Driver/clang-offload-bundler-zlib.c' FAILED ********************
Exit Code: 1

Command Output (stderr):
--
RUN: at line 7: /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang -O0 -target x86_64-unknown-linux-gnu /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/clang-offload-bundler-zlib.c -c -emit-llvm -o /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.bc
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang -O0 -target x86_64-unknown-linux-gnu /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/clang-offload-bundler-zlib.c -c -emit-llvm -o /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.bc
RUN: at line 12: touch /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.empty
+ touch /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.empty
RUN: at line 17: echo 'Content of device file 1' > /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.tgt1
+ echo 'Content of device file 1'
RUN: at line 18: echo 'Content of device file 2' > /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.tgt2
+ echo 'Content of device file 2'
RUN: at line 23: env OFFLOAD_BUNDLER_COMPRESS=1 OFFLOAD_BUNDLER_VERBOSE=1    clang-offload-bundler -type=bc -targets=hip-amdgcn-amd-amdhsa--gfx900,hip-amdgcn-amd-amdhsa--gfx906    -input=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.tgt1 -input=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.tgt2 -output=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.hip.bundle.bc 2>&1 |    /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck -check-prefix=COMPRESS /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/clang-offload-bundler-zlib.c
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck -check-prefix=COMPRESS /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/clang-offload-bundler-zlib.c
+ env OFFLOAD_BUNDLER_COMPRESS=1 OFFLOAD_BUNDLER_VERBOSE=1 clang-offload-bundler -type=bc -targets=hip-amdgcn-amd-amdhsa--gfx900,hip-amdgcn-amd-amdhsa--gfx906 -input=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.tgt1 -input=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.tgt2 -output=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/tools/clang/test/Driver/Output/clang-offload-bundler-zlib.c.tmp.hip.bundle.bc
[1m/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/clang-offload-bundler-zlib.c:59:14: [0m[0;1;31merror: [0m[1mCOMPRESS: expected string not found in input
[0m// COMPRESS: Compressed bundle format version: 3
[0;1;32m             ^
[0m[1m<stdin>:4:21: [0m[0;1;30mnote: [0m[1mscanning from here
[0mCompression level: 6
[0;1;32m                    ^
[0m[1m<stdin>:7:1: [0m[0;1;30mnote: [0m[1mpossible intended match here
[0mCompression rate: 1.86
[0;1;32m^
[0m
Input file: <stdin>
Check file: /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/clang-offload-bundler-zlib.c

-dump-input=help explains the following input dump.

Input was:
<<<<<<
[1m[0m[0;1;30m            1: [0m[1m[0;1;46mCompressed bundle format version: 2 [0m
[0;1;30m            2: [0m[1m[0;1;46mTotal file size (including headers): 125 bytes [0m
[0;1;30m            3: [0m[1m[0;1;46m[0mCompression method used: zlib[0;1;46m [0m
[0;1;32mcheck:36       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;30m            4: [0m[1m[0;1;46m[0mCompression level: 6[0;1;46m [0m
[0;1;32mcheck:37       ^~~~~~~~~~~~~~~~~~~~
[0m[0;1;31mcheck:59'0                         X error: no match found
[0m[0;1;30m            5: [0m[1m[0;1;46mBinary size before compression: 188 bytes [0m
[0;1;31mcheck:59'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;30m            6: [0m[1m[0;1;46mBinary size after compression: 101 bytes [0m
[0;1;31mcheck:59'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;30m            7: [0m[1m[0;1;46mCompression rate: 1.86 [0m
[0;1;31mcheck:59'0     ~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;35mcheck:59'1     ?                       possible intended match
[0m[0;1;30m            8: [0m[1m[0;1;46mCompression ratio: 53.72% [0m
[0;1;31mcheck:59'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~
[0m[0;1;30m            9: [0m[1m[0;1;46mCompression speed: 1.49 MB/s [0m
...

```

</details>

https://github.com/llvm/llvm-project/pull/122307