[llvm-dev] Adding accelerator tables to existing linked DWARF files

Mon Mar 2 15:33:35 PST 2020

Which seems like what we'd want dsymutil to do anyhow?

-eric

On Mon, Mar 2, 2020 at 3:21 PM Greg Clayton via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> On other options would be to make a new "llvm-dwarfld" tool, where most of
> the functionality would exist llvm/lib/DwarfLinker and other locations. The
> idea would be to do any post processing to DWARF using this tool. For
> accelerator tables, it could just create the new sections and then call
> "llvm-objcopy" to add them to the binary.
>
> This tool could eventually be used to optimize DWARF (dead strip code,
> remove unused types, unique types with ODR like llvm-dsymutil, etc).
>
>
> On Mar 2, 2020, at 1:48 PM, Greg Clayton <clayborg at gmail.com> wrote:
>
>
>
> On Feb 28, 2020, at 11:25 PM, Fangrui Song via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> On 2020-02-28, Greg Clayton via llvm-dev wrote:
>
> I am looking to create a tool that can add Apple or DWARF5 accelerator
> tables to fully linked executables that contain DWARF. This will help us
> benchmark how much accelerator tables can improve the debugging experience
> as debuggers don't need to manually index all of the debug info during
> debugging.
>
> Is it for ELF, Mach-O, wasm, COFF, or any of the combinations?
>
>
> Yes, for any object files that LLVM currently supports. But I am looking
> to support ELF first as MachO already has these tables available since
> dsymutil already creates either Apple or DWARF accelerator tables. COFF and
> Wasm can come later.
>
>
> Looking at how accelerator tables are currently emitted, they seem to be
> built up as DWARF is being created or linked, and then emitted using a
> subclass of DWARFEmitter. The only subclass if this right now that I see is
> one in dsymutil which ends up emitting everything using an AsmPrinter by
> eventually emitAppleAccelTable(...) from
> llvm/include/llvm/CodeGen/AccelTable.h.
>
> I spoke briefly with Shoaib on this subject and he suggested adding code
> to llvm-objcopy. I briefly looked through the code and from what I can
> tell, llvm-objcopy doesn't seem to have any DWARF abilities other than
> compressing DWARF sections. If we do add functionality to llvm-objcopy, are
> we ok pulling in DebugInfoDWARF and the LLVM object model? AFAICT the code
> for this tools has its own object file layer which doesn't match the full
> layer inside of LLVM (llvm::ObjectFile and DWARFContext). Also, no
> AsmPrinter objects are used in this codebase either.
>
>
> llvm-objcopy supports various ad-hoc binary manipulation features where
> each feature does a very
> simple task. Neither llvm-objcopy nor GNU objcopy knows DWARF.
> --strip-debug,
> --compress-debug-sections, --add-gnu-debuglink and --only-keep-debug have
> "debug" in their names but
> these features don't need to parse DWARF. (GNU objcopy has a --debugging
> but that only works for
> a.out and coff, not elf).
>
> Do we have a more suitable tool for such debugging functionality? dsymutil
> for ELF?
>
>
> dsymutil is such a tool, but it uses the llvm::ObjectFile layer and the
> llvm targets, so if you open a file that contains "armv7" architecture and
> the ARM target hasn't been built into your bistro, it will fail to open
> this binary with an error that says:
>
> No available targets are compatible with triple "arm-unknown-unknown"
>
> I ran into this with a recent gsym patch that is trying to fix the
> buildbots for testing, but it fails when the ARM targets are not enabled
> and I try to load the DWARF from an object file:
>
>
> http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/63473/steps/test-check-all/logs/stdio
>
> And this is part of the reason for this email. I would love to not require
> llvm-gsymutil to require all LLVM targets to be there. DebugInfoDWARF
> doesn't need the targets, it just needs to know address byte size and
> endianness and it can parse the debug info in the DWARF.
>
> So my main question stands: do we want all tools that must manipulate
> DWARF to require the llvm::ObjectFile layer and all of the targets to be
> enabled just so that the object files can be parsed, or do we want to make
> lighter layer available, akin to what llvm-objcopy has, so more tools can
> take advantage of this lighter weight layer.
>
>
> Looking at lld sources, is seems to use the DebugInfoDWARF library to some
> extent already. Not sure if this tool uses the standard LLVM object model
> or has all of its own emitters. Does lld use AsmPrinter at all? I don't see
> any mention of it in there.
>
>
> --gdb-index and diagnostics (line tables) use DebugInfoDWARF. I have a
> plan to implement .debug_names, which is similar to --gdb-index.
>
>
> That is great, and we will share code for this of course between the tool
> I write and the modifications to lld. Does lld use the llvm::ObjectFile
> layer? or does it have its own lighter weight layer?
>
>
> dsymutil has a --update feature which seems to load all of the DWARF and
> pretend to link it all the while generating the new accelerator table data,
> but I fear using this would pull it way too much code (AsmPrinter, all
> targets required to load all object files types, the standard llvm object
> file model (not the lld or llvm-objcopy versions), targets, etc).
>
> My initial thoughts are:
> 1 - load a DWARFContext and iterate through the DWARF and build
> accelerator table data
> 2 - create the sections for the accelerator tables and either keep in
> memory or save to disk
> 3 - call functions to add the newly created sections to the binary
>
> #1 should be easy as long as I can use a DWARFContext from DebugInfoDWARF.
> #2 might need to be re-implemented using something other than an
> AsmPrinter?
> #3 can use llvm-objcopy code if needed since it can add sections?
>
> Any advice on how this can or should be implemented would be appreciated
> from anyone with experience.
>
> Greg Clayton
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200302/15f934c6/attachment.html>