[llvm] 'Soft Stop' solution on offset overflow issue: By Produceing a truncated but valid DWP file, discarding any DWO files that would not fit within the 32 bit/4GB limits of the format. (PR #71902)

[Jinjie Huang via llvm-commits]

Labman-001 wrote:

> Generally looks OK to me - I guess this isn't practical to test due to the size needed to exceed the limit?

Well, I think it's not totally impossible to test. I have compiled the modified project locally and used **llvm-project/llvm/test/tools/llvm-dwp/Inputs/overflow/debug_info_v4.s and main_v4.s files to generate a .dwp file with debug_info exceeding 4GB.**
Here is how test it:
1. Generate debug_info_v4.dwo(4GB debug_info) and main_v4.dwo (modify to 20M debug_info)
./bin/llvm-mc --triple=x86_64-unknown-linux --filetype=obj --split-dwarf-file=debug_info_v4.dwo -dwarf-version=4 ../llvm/test/tools/llvm-dwp/Inputs/overflow/debug_info_v4.s -o debug_info_v4.o
./bin/llvm-mc --triple=x86_64-unknown-linux --filetype=obj --split-dwarf-file=main_v4.dwo -dwarf-version=4 ../llvm/test/tools/llvm-dwp/Inputs/overflow/main_v4.s -o main_v4.o
2. Generate .dwp file with 'soft-stop'
./bin/llvm-dwp debug_info_v4.dwo main_v4.dwo -continue-on-cu-index-overflow soft-stop -o overflow_soft.dwp
Section Result:
    <img width="738" alt="image" src="https://github.com/llvm/llvm-project/assets/150100070/90dd8058-fbbd-4d97-a08a-406a2830dfe1">
    <img width="351" alt="image" src="https://github.com/llvm/llvm-project/assets/150100070/94f34242-1a1c-4c5f-a523-bea0e6bbda20">
3. Generate .dwp file with 'continue'
./bin/llvm-dwp debug_info_v4.dwo main_v4.dwo -continue-on-cu-index-overflow continue -o overflow.dwp
Section Result:
    <img width="738" alt="image" src="https://github.com/llvm/llvm-project/assets/150100070/6c1403fe-ecde-40f2-a3ff-eb2169e7fb0e">
    <img width="342" alt="image" src="https://github.com/llvm/llvm-project/assets/150100070/29e2ba37-4a67-493a-a14e-76c43bf58c9b">
It can be observed that using the continue option .dwp file increased the size by 20M, and when the soft-stop was enabled, the tool discarded the part that exceeded 4GB.
BTW, we have done some tests based on our business project to generate this kind of 'soft-stop' .dwp files and validated similar results.


https://github.com/llvm/llvm-project/pull/71902