[llvm-dev] Adding accelerator tables to existing linked DWARF files

Tue Mar 3 13:33:51 PST 2020

3/3/2020 12:48 AM, Greg Clayton via llvm-dev пишет:
>
>
>> On Feb 28, 2020, at 11:25 PM, Fangrui Song via llvm-dev 
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>> On 2020-02-28, Greg Clayton via llvm-dev wrote:
>>> I am looking to create a tool that can add Apple or DWARF5 
>>> accelerator tables to fully linked executables that contain DWARF. 
>>> This will help us benchmark how much accelerator tables can improve 
>>> the debugging experience as debuggers don't need to manually index 
>>> all of the debug info during debugging.
>> Is it for ELF, Mach-O, wasm, COFF, or any of the combinations?
>
> Yes, for any object files that LLVM currently supports. But I am 
> looking to support ELF first as MachO already has these tables 
> available since dsymutil already creates either Apple or DWARF 
> accelerator tables. COFF and Wasm can come later.
>
>>
>>> Looking at how accelerator tables are currently emitted, they seem 
>>> to be built up as DWARF is being created or linked, and then emitted 
>>> using a subclass of DWARFEmitter. The only subclass if this right 
>>> now that I see is one in dsymutil which ends up emitting everything 
>>> using an AsmPrinter by eventually emitAppleAccelTable(...) from 
>>> llvm/include/llvm/CodeGen/AccelTable.h.
>>>
>>> I spoke briefly with Shoaib on this subject and he suggested adding 
>>> code to llvm-objcopy. I briefly looked through the code and from 
>>> what I can tell, llvm-objcopy doesn't seem to have any DWARF 
>>> abilities other than compressing DWARF sections. If we do add 
>>> functionality to llvm-objcopy, are we ok pulling in DebugInfoDWARF 
>>> and the LLVM object model? AFAICT the code for this tools has its 
>>> own object file layer which doesn't match the full layer inside of 
>>> LLVM (llvm::ObjectFile and DWARFContext). Also, no AsmPrinter 
>>> objects are used in this codebase either.
>>
>> llvm-objcopy supports various ad-hoc binary manipulation features 
>> where each feature does a very
>> simple task. Neither llvm-objcopy nor GNU objcopy knows DWARF. 
>> --strip-debug,
>> --compress-debug-sections, --add-gnu-debuglink and --only-keep-debug 
>> have "debug" in their names but
>> these features don't need to parse DWARF. (GNU objcopy has a 
>> --debugging but that only works for
>> a.out and coff, not elf).
>>
>> Do we have a more suitable tool for such debugging functionality? 
>> dsymutil for ELF?
>
> dsymutil is such a tool, but it uses the llvm::ObjectFile layer and 
> the llvm targets, so if you open a file that contains "armv7" 
> architecture and the ARM target hasn't been built into your bistro, it 
> will fail to open this binary with an error that says:
>
> No available targets are compatible with triple "arm-unknown-unknown"
>
> I ran into this with a recent gsym patch that is trying to fix the 
> buildbots for testing, but it fails when the ARM targets are not 
> enabled and I try to load the DWARF from an object file:
>
> http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/63473/steps/test-check-all/logs/stdio
>
> And this is part of the reason for this email. I would love to not 
> require llvm-gsymutil to require all LLVM targets to be there. 
> DebugInfoDWARF doesn't need the targets, it just needs to know address 
> byte size and endianness and it can parse the debug info in the DWARF.
>
Hi Greg,  dsymutil code was recently refactored so that it`s linking 
part was moved into lib/DWARFLinker library.

As an advantage, it allows making DWARFLinker to not depend on 
ObjectFile : https://reviews.llvm.org/D75029 .

It might make sense that your tool uses DWARFLinker library(probably 
extending it with new functionality) to work with DWARF.

Alexey.

> So my main question stands: do we want all tools that must manipulate 
> DWARF to require the llvm::ObjectFile layer and all of the targets to 
> be enabled just so that the object files can be parsed, or do we want 
> to make lighter layer available, akin to what llvm-objcopy has, so 
> more tools can take advantage of this lighter weight layer.
>
>>
>>> Looking at lld sources, is seems to use the DebugInfoDWARF library 
>>> to some extent already. Not sure if this tool uses the standard LLVM 
>>> object model or has all of its own emitters. Does lld use AsmPrinter 
>>> at all? I don't see any mention of it in there.
>>
>> --gdb-index and diagnostics (line tables) use DebugInfoDWARF. I have 
>> a plan to implement .debug_names, which is similar to --gdb-index.
>
> That is great, and we will share code for this of course between the 
> tool I write and the modifications to lld. Does lld use the 
> llvm::ObjectFile layer? or does it have its own lighter weight layer?
>>
>>> dsymutil has a --update feature which seems to load all of the DWARF 
>>> and pretend to link it all the while generating the new accelerator 
>>> table data, but I fear using this would pull it way too much code 
>>> (AsmPrinter, all targets required to load all object files types, 
>>> the standard llvm object file model (not the lld or llvm-objcopy 
>>> versions), targets, etc).
>>>
>>> My initial thoughts are:
>>> 1 - load a DWARFContext and iterate through the DWARF and build 
>>> accelerator table data
>>> 2 - create the sections for the accelerator tables and either keep 
>>> in memory or save to disk
>>> 3 - call functions to add the newly created sections to the binary
>>>
>>> #1 should be easy as long as I can use a DWARFContext from 
>>> DebugInfoDWARF.
>>> #2 might need to be re-implemented using something other than an 
>>> AsmPrinter?
>>> #3 can use llvm-objcopy code if needed since it can add sections?
>>>
>>> Any advice on how this can or should be implemented would be 
>>> appreciated from anyone with experience.
>>>
>>> Greg Clayton
>>>
>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200304/d585eb55/attachment.html>