[lldb-dev] Are overlapping ELF sections problematic?

[Pavel Labath via lldb-dev]

On 04/06/2019 11:21, Thomas Goodfellow wrote:
> Hi Pavel
> 
>> I can't say what's the situation in the rest of llvm, but right now lldb
>> has zero test coverage for the flow you are using, so the fact that this
>> has worked until now was pretty much an accident.
> 
> It was a pleasant surprise that it worked at all, since flat memory
> maps have become near-ubiquitous. But it's good to at least know that
> the conceptual ice hasn't become any thinner through the patch, i.e.
> it refines the existing state rather than reflecting a more explicit
> policy change.

Yes, I didn't mean to make anything drastic with this patch. However, I 
would say that independently of this patch, in the past few years, lldb 
has gotten more strict in accepting features/fixes which don't have test 
coverage and/or are useful in only some peculiar downstream use case 
(see removal of ocaml/go/java language support, etc.)..

> 
>> In the mean time, I believe you can just patch out the part which drops
>> the overlapping sections from the section list and get behavior which
>> was more-or-less identical to the old one.
> 
> I think this also requires reverting the use of the IntervalMap as the
> VM address container, since that relies upon non-overlapping
> intervals? That smells like a bigger fork than I would want like to
> keep indefinitely alive.

It sounds like you might be able to just skip adding some (all?) of the 
sections into the interval map, which should result in all of them being 
created, like they used to be.

Or maybe you could fudge their "file addresses" and remap them into 
non-overlapping regions at this level too. It would break lookups by 
file addresses for the remapped sections, but this is something that 
didn't work already when the addresses overlapped. I'm not sure what 
else could be broken by this.. We already do some fudging like this for 
relocatable (.o) files, which have all addresses starting at zero, so it 
seems like at least something can work here.

For my own education, would you be able to send me one of your files 
with these overlapping sections (or maybe just the output of "readelf 
-e" or something)? I don't know much about these more exotic platforms, 
so being aware things like these might be of help when doing future changes.

Incidentally, I was just made aware that this change also breaks for 
thread-local sections, which can appear to have overlapping file 
addresses with other sections. So I will probably be revisiting this 
piece of code soon. However, right not my thinking is to simply stop 
putting thread-local section address range map while simultaneously 
starting to ignore them for file address lookups (as thread-local 
sections need to be handled in a more complex manner anyway). This won't 
help your use case much...

> 
>> I believe that a long term solution here would be to introduce some
>> concept of address spaces to lldb. Then these queries would no longer be
>> ambiguous as the function FindSectionContainingFileAddress would
>> (presumably) take an additional address-space identifier as an argument.
>> I know this is what some downstream users are doing to make things like
>> this work. However, this is a fairly invasive change, so doing something
>> like this upstream would require a lot of previous discussion.
> 
> Would this also extend the GDB remote protocol, where the single flat
> address space seems the only current option? (at least the common
> solution in various GDB discussions of DSP targets is address muxing
> of the sort we're using)

I would say "hopefully yes", but I not very familiar with these kinds of 
targets.

> 
> I imagine such changes are hampered by the lack of in-tree targets
> that require them, both to motivate the change and to keep it testable
> (the recent "removing magic numbers assuming 8-bit bytes" discussion
> in llvm-dev features the same issue). Previously Embecosm was
> attempting to upstream a LLVM target for its demonstration AAP
> architecture (features multiple address spaces), e.g.
> http://lists.llvm.org/pipermail/llvm-dev/2017-February/109776.html .
> However their public forks on GitHub only reveal GDB support rather
> than LLDB, and that implementation is by an address mux.
> 
> Unfortunately the architecture I'm working with is (yet another) poor
> candidate for upstreaming, since it lacks general availability, but
> hopefully one of the exotic architectures lurking in the LLVM shadows
> someday steps forth with a commitment to keep it alive in-tree.
> 

Yeah, the lack of in-tree targets is one of the causes (but also a 
consequence of ?) the lack of address space support. I've been following 
the non-8-bit thread from a distance, and FWIW, I would be fine with 
having some kind of a mock target supporting these things in lldb. I 
might even prefer debugging things against a simple mock instead of some 
complicated-but-real target.

The other causes are the main contributors not knowing enough about 
these architectures to help drive this, and just being generally busy 
with other stuff. :/

cheers,
pavel