[llvm-dev] Switch to ld.bfd tombstone behavior by default

Fri Jul 24 06:21:56 PDT 2020

>>
>> >> Though it looks like there still exist case when range list could be terminated earlier:
>> >>
>> >> base address selection entry: {-1, address of deleted code}
>> >> following range list entry: {0, 0} << points to the same address as set by base address 
>selection entry and has zero size.
>> >
>> >That's a bug in the producer (though a good point - I've probably made
>> >that bug in LLVM) - the linker can't solve that problem, since the
>> >linker can't touch the literal unrelocated 0, 0.
>> >
>> >> after linker resolved relocations it would look like this, for bfd case:
>> >>
>> >> base address selection entry: {-1, 1}
>> >> following range list entry: {0, 0} <<<<<<<<<<<
>> >>
>> >> So there still exists {0,0} entry which could be considered as the end of list entry.
>> >>
>> >> But old lld solution has the same problem, thus it would not be new.
>> >>
>> >> - Additionally, AFAIK gdb has special processing for overlapped address
>> >> ranges starting from 0. Using bfd tombstone value could break that processing - I would check it.
>> >
>> >Not sure I understand - presumably gdb's special processing is
>> >intended to work with bfd's tombstoning, since it's been the most
>> >common/prolific unix linker, the one intended to work with gdb (they
>> >exist in the same repository) for decades, right?
>>
>> I think I misunderstood this: "I wonder how well the bfd tombstoning works in DWARFv5
>> (rnglists/loclists)". I read it as we would like to use bfd tombstoning(1 for ranges)
>> for rnglists/loclists also.
>>
>> So, right. bfd's tombstoning works correctly with gdb until 1 is not used
>> for rnglists/loclists as tombstone value.
>
>Not quite parsing this, but I think we're on the same page - that
>bfd's tombstoning "1 for debug_ranges/debug_loc, 0 (not 0+addend, but
>absolute 0) for everything else (including debug_loclists and
>debug_rnglists)" is probably the most likely to work for gdb, since
>it's been deployed for a long time.
>

right.

Speaking of our needs - it would be enough to have a command line option 
to switch -1/-2 tombstoning on, while bfd tombstoning would be default.

>- Dave
>
>PS: Fair point about base address specifiers being able to produce 0,0
>entries - wouldn't mind fixing that in LLVM if I knew of an easy way
>to test at compile-time whether the difference between two labels was
>zero, then skip that entry in the lists entirely. Would save space and
>address the original issue I had with debug_ranges terminating early.
>(should've thought about that much earlier than my more alarmist "oh
>deer, we need to fundamentally change how linkers resolve relocations
>because everything's been broken and we just didn't realize it" -
>fixing the compiler not to produce zero-length ranges would've been
>less risky & probably still worth doing - though addressing the
>broader issue to help with your situation of 0 as a valid address I
>think is still good too)
>

agreed. It would be the most proper way to avoid zero-length ranges 
at generation time if possible.

side note - another possibility is the optimizing tool like dsymutil which could remove 
zero length address ranges. It would not help to correct parsing but 
would save space. 

Alexey.

>
> > > >> With consideration for satefy for the upcoming release/11.x, we can make
> > > >> two choices:
> > > >>
> > > >> a) .debug_ranges&.debug_loc => -2, .debug_line => 0, other .debug_* -> -1
> > > >> b) .debug_ranges&.debug_loc => -2, other .debug_* => 0
> > > >>
> > > >> Delaying .debug_line => -1 for one or two release sounds good to me.
> > > >> So LLD 11 or 12 linked binaries can be debugged by LLDB 10. This is a
> > > >> nice property.
> > > >>
> > > >> This write-up proposes b), but I'd say a) is likely sufficient. With the
> > > >> available information, I cannot yet say that a) will have more risk.
> > > >
> > > >Risk is about the unknowns - and it still seems like a lot of
> > > >unknowns. While there are probably many more consumers that read
> > > >.debug_line than other sections, reading debug_info (for instance) is
> > > >necessary for inline frames in symbolizing - still probably one of the
> > > >most common uses of DWARF I'd guess. (what about stack unwinding using
> > > >debug_frame? that'd worry me a bit if anyone got /that/ wrong because
> > > >of this change)
> > > >
> > > >> > - chromium/firefox have some tools that were broken:
> > > >> > https://bugs.chromium.org/p/chromium/issues/detail?id=1102223#c5
> > > >>
> > > >> This is potentially related to other .debug_* (not .debug_line)
> > > >> I hope Chromium developers can chime in here:) The breakage was
> > > >> unfortunate but I don't know how we could have avoided that. IMHO this
> > > >> is no different from "clang started to emit a new DW_FORM_* and a
> > > >> postprocessing tool of .debug chokes on that" Whether we want to
> > > >> suppress that particular DW_FORM_* definitely should depend on how
> > > >> likely it can cause problems, but we can't yet say we have to hold off
> > > >> on a feature for a solved (precisely, mitigated) problem.
> > > >
> > > >LLVM has no custom forms and I'd be super cautious about adding any
> > > >that were on by default because of how bad that breakage would be.
> > > >
> > > >I'm not so concerned about the problems we know - but what they tell
> > > >us about the problems that might arise from use cases we don't know.
> > > >All the other projects out there that might have custom DWARF parsers
> > > >to do some ad-hoc things.
> > > >
> > > >(also, ultimately - given how far-reaching this is, I think we'll want
> > > >some tidier flags that are more user-focussed. I'd hope for a flag
> > > >that gives BFD-like semantics (though I'd be OK with fixing debug_loc
> > > >(using 1 instead of 0) to work the same as debug_ranges while we're
> > > >there - a minor divergence from BFD, but highly likely to not cause
> > > >problems/fall out naturally from a simple implementation of parsing
> > > >that section) - something that's been in-use and tested by basically
> > > >everyone for decades. And another flag for the new semantics (-2 for
> > > >debug_loc/debug_ranges, -1 everywhere else). Customizable per-section
> > > >expression-based support I think is a recipe for platform divergence &
> > > >I'd rather it not be available/supported at all, but if you really
> > > >want to keep it in, I'd at least rather it not be the feature we
> > > >promote to users about how they can test/opt in/out of the behavior
> > > >when they're seeing breakages or want to test the future semantics)
> > > >
> > > >> >I'm not sure how to get the word out to DWARF consumers that they should
> > > >> >consider this new experimental behavior. Ray's done a good job
> > > >> >evangelizing/discussing this with gdb and lldb at least - and of course
> > > >> >having turned it on by default briefly has found some users (like Chromium)
> > > >> >that we probably wouldn't have found no matter how long we left this as an
> > > >> >experimental option... so some things are going to break when we switch no
> > > >> >matter what.
> > > >>
> > > >> Thank you for following up with some GNU folks on their lists!
> > > >> If folks want to follow along the thread:
> > > >> https://sourceware.org/pipermail/binutils/2020-June/111376.html
> > > >>
> > > >> We have informed binutils, elfutils-devel (elfutils has a few debug
> > > >> tools) and gdb. I don't recall that anyone has thought about problems
> > > >> with a tombstone value.
> > > >>
> > > >> >
> > > >> >P.S: Sony's already been using the -1 technique with their debugger and
> > > >> >linker for a while, so they may want to keep this on by default for SCE -
> > > >> >but I'm not sure how to do that in-tree.
> > > >> >
> > > >> >
> > > >> >Clang doesn't know which lld
> > > >> >version it's running, so whether the flag can be specified, I would think?
> > > >> >(so it'd be hard to have Clang go "if SCE and LLD, pass the flag to use
> > > >> >-1", I think) - if there is a way to make that decision in the compiler
> > > >.> >driver+linker, then we'd have a question of "default new behavior except
> > > >> >when tuning for LLDB and GDB" or "default bfd behavior except when tuning
> > > >> >for SCE".
> > > >>
> > > >> I've been involed in another thread on SHF_LINK_ORDER (https://sourceware.org/pipermail/binutils/2020-
> > > >July/112415.html ).
> > > >> We may need a way to tell codegen about the used linker.
> > > >>
> > > >> pcc proposed -mbinutils-version= - This is nice in that some MC
> > > >> decisions related to -fno-integrated-as can use this option as well.
> > > >> jyknight proposed -mlinker-version= and syntax like -fuse-ld=bfd:2.34
> > > >>
> > > >> This may get more complex if the generated object file want to be linked
> > > >> with more than one linker. This discussion probably deserves its own
> > > >> thread.
> > >
> > >
> > > --
> > > Alexey Lapshin
> > >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev