[LLVMdev] RFC: Using zlib to decompress debug info sections.

[Eric Christopher]

Historically we've done the former. The latter would require Chris
wanting to do that.

-eric

On Tue, Apr 16, 2013 at 11:52 AM, Alexey Samsonov <samsonov at google.com> wrote:
> Just in case - do we want to link with libz.so installed in the system, or
> be self-contained and copy sources to LLVM repo?
>
>
> On Tue, Apr 16, 2013 at 10:48 PM, Eric Christopher <echristo at gmail.com>
> wrote:
>>
>> On Tue, Apr 16, 2013 at 9:37 AM, Alexey Samsonov <samsonov at google.com>
>> wrote:
>> >
>> > On Tue, Apr 16, 2013 at 8:31 PM, Michael Spencer <bigcheesegs at gmail.com>
>> > wrote:
>> >>
>> >> On Tue, Apr 16, 2013 at 2:37 AM, Alexey Samsonov <samsonov at google.com>
>> >> wrote:
>> >>>
>> >>> Hi!
>> >>>
>> >>> TL;DR WDYT of adding zlib decompression capabilities to LLVMObject
>> >>> library?
>> >>>
>> >>>
>> >>> ld.gold from GNU binutils has --compress-debug-sections=zlib option,
>> >>> which uses zlib to compress .debug_xxx sections and renames them to
>> >>> .zdebug_xxx.
>> >>> binutils (and GDB) support this properly, while LLVM command line
>> >>> tools
>> >>> don't:
>> >>>
>> >>> $ ld --version
>> >>> GNU gold (GNU Binutils for Ubuntu 2.22) 1.11
>> >>> $ ./bin/clang++ -g a.cc -Wl,--compress-debug-sections=zlib
>> >>> $ objdump -h a.out | grep debug
>> >>>  26 .debug_info   00000066  0000000000000000  0000000000000000
>> >>> 00002010
>> >>> 2**0
>> >>>  27 .debug_abbrev 00000048  0000000000000000  0000000000000000
>> >>> 00002068
>> >>> 2**0
>> >>>  28 .debug_aranges 00000000  0000000000000000  0000000000000000
>> >>> 000020bb
>> >>> 2**0
>> >>>  29 .debug_macinfo 00000000  0000000000000000  0000000000000000
>> >>> 000020cf
>> >>> 2**0
>> >>>  30 .debug_line   00000053  0000000000000000  0000000000000000
>> >>> 000020e3
>> >>> 2**0
>> >>>  31 .debug_loc    00000000  0000000000000000  0000000000000000
>> >>> 0000213e
>> >>> 2**0
>> >>>  32 .debug_pubtypes 00000000  0000000000000000  0000000000000000
>> >>> 00002152  2**0
>> >>>  33 .debug_str    00000069  0000000000000000  0000000000000000
>> >>> 00002166
>> >>> 2**0
>> >>>  34 .debug_ranges 00000000  0000000000000000  0000000000000000
>> >>> 000021d9
>> >>> 2**0
>> >>> $ ./bin/llvm-objdump -h a.out | grep debug
>> >>>  27 .zdebug_info  00000058 0000000000000000
>> >>>  28 .zdebug_abbrev 00000053 0000000000000000
>> >>>  29 .zdebug_aranges 00000014 0000000000000000
>> >>>  30 .zdebug_macinfo 00000014 0000000000000000
>> >>>  31 .zdebug_line  0000005b 0000000000000000
>> >>>  32 .zdebug_loc   00000014 0000000000000000
>> >>>  33 .zdebug_pubtypes 00000014 0000000000000000
>> >>>  34 .zdebug_str   00000073 0000000000000000
>> >>>  35 .zdebug_ranges 00000014 0000000000000000
>> >>>
>> >>> Decompression and proper handling of debug info sections may be needed
>> >>> in llvm-dwarfdump and llvm-symbolizer tools. We can implement this by:
>> >>> 1) Checking if zlib is present in the system during configuration.
>> >>> 2) Adding zlib decompression to llvm::MemoryBuffer, and section
>> >>> decompression to LLVMObject (this would require optional linking with
>> >>> -lz).
>> >>> 3) Using the methods in LLVM tools where needed.
>> >>>
>> >>> Does this make sense to you?
>> >>>
>> >>> --
>> >>> Alexey Samsonov, MSK
>> >>
>> >>
>> >> I'm not sure MemoryBuffer is the right place to do this either. I'm
>> >> also
>> >> not sure if we want debug info decompression to be transparent in
>> >> LLVMObject
>> >> or not. I'm leaning towards no since it's not part of the standard yet,
>> >
>> >
>> > Yeah, I also think that decompression should be explicitly requested by
>> > the
>> > user of LLVMObject.
>> >
>> >>
>> >> unless gold is actually using the SHF_COMPRESSED flag.
>> >>
>> >>
>> >> I think it should be part of Object, but as an external API that is
>> >> used
>> >> when you find a section you know from external factors (the name
>> >> matches
>> >> some list) is compressed.
>> >>
>>
>> Definitely want the feature :)
>>
>> I don't see SHF_COMPRESSED (unless readelf just isn't showing it to
>> me), but it wouldn't be too hard to get binutils to mark them as such.
>> Right now the convention is .z<foo> are compressed, but that's not as
>> precise as we'd like it to be. There's been some talk on the binutils
>> list about it, but it hasn't been implemented yet.
>>
>> -eric
>
>
>
>
> --
> Alexey Samsonov, MSK