[llvm-dev] Adding accelerator tables to existing linked DWARF files

Mon Mar 2 15:44:45 PST 2020

I'd like it... Adrian? Fred?

-eric

On Mon, Mar 2, 2020 at 3:44 PM Greg Clayton <clayborg at gmail.com> wrote:

> Yes. I am fine with adding ELF support to llvm-dsymutil if that is the way
> people think we should go?
>
> On Mar 2, 2020, at 3:33 PM, Eric Christopher <echristo at gmail.com> wrote:
>
> Which seems like what we'd want dsymutil to do anyhow?
>
> -eric
>
> On Mon, Mar 2, 2020 at 3:21 PM Greg Clayton via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On other options would be to make a new "llvm-dwarfld" tool, where most
>> of the functionality would exist llvm/lib/DwarfLinker and other locations.
>> The idea would be to do any post processing to DWARF using this tool. For
>> accelerator tables, it could just create the new sections and then call
>> "llvm-objcopy" to add them to the binary.
>>
>> This tool could eventually be used to optimize DWARF (dead strip code,
>> remove unused types, unique types with ODR like llvm-dsymutil, etc).
>>
>>
>> On Mar 2, 2020, at 1:48 PM, Greg Clayton <clayborg at gmail.com> wrote:
>>
>>
>>
>> On Feb 28, 2020, at 11:25 PM, Fangrui Song via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> On 2020-02-28, Greg Clayton via llvm-dev wrote:
>>
>> I am looking to create a tool that can add Apple or DWARF5 accelerator
>> tables to fully linked executables that contain DWARF. This will help us
>> benchmark how much accelerator tables can improve the debugging experience
>> as debuggers don't need to manually index all of the debug info during
>> debugging.
>>
>> Is it for ELF, Mach-O, wasm, COFF, or any of the combinations?
>>
>>
>> Yes, for any object files that LLVM currently supports. But I am looking
>> to support ELF first as MachO already has these tables available since
>> dsymutil already creates either Apple or DWARF accelerator tables. COFF and
>> Wasm can come later.
>>
>>
>> Looking at how accelerator tables are currently emitted, they seem to be
>> built up as DWARF is being created or linked, and then emitted using a
>> subclass of DWARFEmitter. The only subclass if this right now that I see is
>> one in dsymutil which ends up emitting everything using an AsmPrinter by
>> eventually emitAppleAccelTable(...) from
>> llvm/include/llvm/CodeGen/AccelTable.h.
>>
>> I spoke briefly with Shoaib on this subject and he suggested adding code
>> to llvm-objcopy. I briefly looked through the code and from what I can
>> tell, llvm-objcopy doesn't seem to have any DWARF abilities other than
>> compressing DWARF sections. If we do add functionality to llvm-objcopy, are
>> we ok pulling in DebugInfoDWARF and the LLVM object model? AFAICT the code
>> for this tools has its own object file layer which doesn't match the full
>> layer inside of LLVM (llvm::ObjectFile and DWARFContext). Also, no
>> AsmPrinter objects are used in this codebase either.
>>
>>
>> llvm-objcopy supports various ad-hoc binary manipulation features where
>> each feature does a very
>> simple task. Neither llvm-objcopy nor GNU objcopy knows DWARF.
>> --strip-debug,
>> --compress-debug-sections, --add-gnu-debuglink and --only-keep-debug have
>> "debug" in their names but
>> these features don't need to parse DWARF. (GNU objcopy has a --debugging
>> but that only works for
>> a.out and coff, not elf).
>>
>> Do we have a more suitable tool for such debugging functionality?
>> dsymutil for ELF?
>>
>>
>> dsymutil is such a tool, but it uses the llvm::ObjectFile layer and the
>> llvm targets, so if you open a file that contains "armv7" architecture and
>> the ARM target hasn't been built into your bistro, it will fail to open
>> this binary with an error that says:
>>
>> No available targets are compatible with triple "arm-unknown-unknown"
>>
>> I ran into this with a recent gsym patch that is trying to fix the
>> buildbots for testing, but it fails when the ARM targets are not enabled
>> and I try to load the DWARF from an object file:
>>
>>
>> http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/63473/steps/test-check-all/logs/stdio
>>
>> And this is part of the reason for this email. I would love to not
>> require llvm-gsymutil to require all LLVM targets to be there.
>> DebugInfoDWARF doesn't need the targets, it just needs to know address byte
>> size and endianness and it can parse the debug info in the DWARF.
>>
>> So my main question stands: do we want all tools that must manipulate
>> DWARF to require the llvm::ObjectFile layer and all of the targets to be
>> enabled just so that the object files can be parsed, or do we want to make
>> lighter layer available, akin to what llvm-objcopy has, so more tools can
>> take advantage of this lighter weight layer.
>>
>>
>> Looking at lld sources, is seems to use the DebugInfoDWARF library to
>> some extent already. Not sure if this tool uses the standard LLVM object
>> model or has all of its own emitters. Does lld use AsmPrinter at all? I
>> don't see any mention of it in there.
>>
>>
>> --gdb-index and diagnostics (line tables) use DebugInfoDWARF. I have a
>> plan to implement .debug_names, which is similar to --gdb-index.
>>
>>
>> That is great, and we will share code for this of course between the tool
>> I write and the modifications to lld. Does lld use the llvm::ObjectFile
>> layer? or does it have its own lighter weight layer?
>>
>>
>> dsymutil has a --update feature which seems to load all of the DWARF and
>> pretend to link it all the while generating the new accelerator table data,
>> but I fear using this would pull it way too much code (AsmPrinter, all
>> targets required to load all object files types, the standard llvm object
>> file model (not the lld or llvm-objcopy versions), targets, etc).
>>
>> My initial thoughts are:
>> 1 - load a DWARFContext and iterate through the DWARF and build
>> accelerator table data
>> 2 - create the sections for the accelerator tables and either keep in
>> memory or save to disk
>> 3 - call functions to add the newly created sections to the binary
>>
>> #1 should be easy as long as I can use a DWARFContext from DebugInfoDWARF.
>> #2 might need to be re-implemented using something other than an
>> AsmPrinter?
>> #3 can use llvm-objcopy code if needed since it can add sections?
>>
>> Any advice on how this can or should be implemented would be appreciated
>> from anyone with experience.
>>
>> Greg Clayton
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200302/1ca53b04/attachment.html>