[Lldb-commits] [PATCH] D48393: Make DWARFParsing more thread-safe

[Greg Clayton via lldb-commits]

> On Jun 21, 2018, at 7:58 AM, Zachary Turner <zturner at google.com> wrote:
> 
> Performance i get. Gdb is almost unusable for large programs because of how long it takes to load debug info.

Agreed. With our new DWARF5 tables, we will be even better. Or test on a mac with dSYM files and you will get similar numbers to a DWARF5 scenario with .debug_names.

> 
> Do you have specific numbers on memory usage? How much memory (absolute and %) is saved by loading debug info lazily on a relatively large project?

I ran the number many years back when LLDB was first trying to overtake GDB. The numbers were quite favorable at the time. You can easily run a few tests by doing the same thing in LLDB and in GDB like:
- b main.cpp:123
- run
- bt
- frame variable

Then look at the memory numbers of each process.

> On Thu, Jun 21, 2018 at 7:54 AM Greg Clayton <clayborg at gmail.com <mailto:clayborg at gmail.com>> wrote:
> 
> 
>> On Jun 21, 2018, at 7:47 AM, Zachary Turner <zturner at google.com <mailto:zturner at google.com>> wrote:
>> 
>> Related question: Is the laziness done to save memory, startup time, or both?
> 
> Both. It allows us to fetch only what we need when we need it. Time to break at main.cpp:123 is much quicker. Using LLDB for symbolication is much quicker as symbolication only needs to know about function definitions and function bounds. Many uses of LLDB are made better by partially parsing.
> 
>> On Thu, Jun 21, 2018 at 7:36 AM Greg Clayton via Phabricator <reviews at reviews.llvm.org <mailto:reviews at reviews.llvm.org>> wrote:
>> clayborg added a comment.
>> 
>> In https://reviews.llvm.org/D48393#1138989 <https://reviews.llvm.org/D48393#1138989>, @labath wrote:
>> 
>> > I am not sure this will actually solve the problems you are seeing. This may avoid corrupting the internal DenseMap data structures, but it will not make the algorithm using them actually correct.
>> >  For example the pattern in `ParseTypeFromDWARF` is:
>> >
>> > 1. check the "already parsed map". If the DIE is already parsed then you're done.
>> > 2. if the map contains the magic "DIE_IS_BEING_PARSED" key, abort (recursive dwarf references)
>> > 3. otherwise, insert the  "DIE_IS_BEING_PARSED" key into the map
>> > 4. do the parsing, which potentially involves recursive `ParseTypeFromDWARF` calls
>> > 5. insert the parsed type into the map
>> >
>> >   What you do is make each of the steps (1), (3), (5) atomic individually. However, the whole algorithm is not correct unless the whole sequence is atomic as a whole. Otherwise, if you have two threads trying to parse the same DIE (directly or indirectly), one of them could see the intermediate DIE_IS_BEING_PARSED and incorrectly assume that it encountered recursive types.
>> 
>> 
>> We need to make #1 atomic.
>> #2 would need to somehow know if the type is already being parsed recursively by the current thread. If so, then do what we do now. If not, we need a way to wait on the completion of this type so the other parsing thread can complete it and put it into the map, at which time we grab the right value from the map
>> So #6 step would need to be added so after we do put it into the map, we can notify other threads that might be waiting
>> 
>> > So, I think that locking at a higher level would be better. Doing that will certainly be tricky though...
>> 
>> 
>> 
>> 
>> https://reviews.llvm.org/D48393 <https://reviews.llvm.org/D48393>
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20180621/32548f04/attachment.html>