[LLVMdev] Looking for a fix to memory leak in DWARF support

[Eric Christopher]

On Fri, Jun 13, 2014 at 4:02 PM, Sergei Larin <slarin at codeaurora.org> wrote:
> Eric,
>
>   Let me clarify it a bit...  without type uniqueing for LTO + debug will I have a highly inefficient IR representation or incorrect debug info? If debug info for LTO is known to be non-useful or ambiguous or flat wrong -  there is no point in fixing its emission... or will it still be practical and if I manage to improve it somewhat the customer will still have some value-add by using it?
>

In current ToT it should be correct. In 3.1? It'll be, at best,
inefficient - likely a big pile of bugs to fix to get that to work
will also fix any problems that show up.

That said, it's a really bad idea to try to do it.

-eric

> Thanks...
>
> Sergei
>
> ---
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
>
>> -----Original Message-----
>> From: Eric Christopher [mailto:echristo at gmail.com]
>> Sent: Friday, June 13, 2014 4:32 PM
>> To: Sergei Larin
>> Cc: David Blaikie; LLVM Developers Mailing List
>> Subject: Re: [LLVMdev] Looking for a fix to memory leak in DWARF support
>>
>> On Fri, Jun 13, 2014 at 2:28 PM, Sergei Larin <slarin at codeaurora.org> wrote:
>> >
>> > Thanks Eric,
>> >
>> >
>> > They are doing LTO build but with some custom modifications (think a
>> library at a time as opposed to a whole program). I must admit, it is a rather
>> large application as well, so as expected, any inefficiencies are multiplied
>> greatly.
>> >
>> > From little that I have seen so far, it looks like debug metadata for an IR
>> object linger behind once the object itself is eliminated (optimized). Rings a
>> bell?
>> >
>>
>> They're going to run into all sorts of problems. They're doomed if it's a large
>> C++ application (or really even if it's a C based application).
>>
>> They won't get this to link likely with LTO + debug. It's unlikely to be a
>> memory leak. It's just an explosion of debug info because of our lack of
>> uniquing for type information. You can see my EuroLLVM 2013 talk where I
>> describe this and Manman's patches to help fix the issue over the last couple
>> of years.
>>
>> -eric
>>
>> > Thanks.
>> >
>> > Sergei
>> >
>> > ---
>> > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>> > hosted by The Linux Foundation
>> >
>> >
>> >> -----Original Message-----
>> >> From: Eric Christopher [mailto:echristo at gmail.com]
>> >> Sent: Friday, June 13, 2014 4:00 PM
>> >> To: Sergei Larin
>> >> Cc: David Blaikie; LLVM Developers Mailing List
>> >> Subject: Re: [LLVMdev] Looking for a fix to memory leak in DWARF
>> >> support
>> >>
>> >> On Fri, Jun 13, 2014 at 11:03 AM, Sergei Larin <slarin at codeaurora.org>
>> wrote:
>> >> >
>> >> > David,
>> >> >
>> >> > Thanks for the quick response...
>> >> >
>> >> >
>> >> > No, at this point I am just getting into the issue... I assume it
>> >> > is a leak, but
>> >> no clear proof yet. I was hoping it was an obvious thing since I
>> >> recall a discussion about it a while ago... but maybe I am just confused.
>> >> >
>> >> > Was your work for compressing DWARF data  motivated by a certain
>> >> inefficiency in debug info representation? Did it result in significant
>> savings?
>> >> >
>> >>
>> >> The compression isn't related to the IR level and some data is here
>> >> on that
>> >> compressing:
>> >>
>> >> https://gcc.gnu.org/wiki/DebugFission
>> >>
>> >> The internal representation and code have change significantly since
>> >> then and there's no real way to help much. It may be a memory leak,
>> >> but I don't recall one that bad back then. How is your customer compiling?
>> LTO?
>> >> Something else?
>> >>
>> >> -eric
>> >>
>> >> > Thanks.
>> >> >
>> >> > Sergei
>> >> >
>> >> > ---
>> >> > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>> >> > hosted by The Linux Foundation
>> >> >
>> >> >
>> >> >> -----Original Message-----
>> >> >> From: David Blaikie [mailto:dblaikie at gmail.com]
>> >> >> Sent: Friday, June 13, 2014 12:41 PM
>> >> >> To: Sergei Larin
>> >> >> Cc: LLVM Developers Mailing List
>> >> >> Subject: Re: Looking for a fix to memory leak in DWARF support
>> >> >>
>> >> >> Are you sure it's a leak, not just a large memory footprint?
>> >> >>
>> >> >> A lot of things have changed. Heck, even the bugs that we've fixed
>> >> >> might not have existed two years ago (I've certainly created (and
>> >> >> fixed) my share of bugs in debug info).
>> >> >>
>> >> >> Best you start with a leak tracking tool and at least be able to
>> >> >> point to the major leak culprit (eg: "90% of the leaked memory was
>> >> >> allocated at <stack
>> >> >> trace>")
>> >> >>
>> >> >> On Fri, Jun 13, 2014 at 9:16 AM, Sergei Larin
>> >> >> <slarin at codeaurora.org>
>> >> wrote:
>> >> >> >
>> >> >> > David, (and everyone else)
>> >> >> >
>> >> >> >   I am forced to do some maintenance work on a fairly old LLVM
>> >> >> > branch (likely based on release 3.1) that among other issues has
>> >> >> > a major problem with memory leak somewhere around DWARF
>> debug
>> >> support.
>> >> >> > In fact customer is unable to build with -g at all - simply
>> >> >> > running out of memory on their project...
>> >> >> >
>> >> >> >   I seem to remember that there has been a major fix related to
>> >> >> > it, but finding it hard to pinpoint. So the appeal to the
>> >> >> > communal memory
>> >> >> > - do you remember if  there was a single fix (since the middle
>> >> >> > of
>> >> >> > 2012 or so) that would have addressed such issue?
>> >> >> >
>> >> >> >   I know the code was drastically change since - I have seen
>> >> >> > this for instance
>> >> >> > llvm.org/viewvc/llvm-
>> project/llvm/trunk/lib/MC/ELFObjectWriter.cpp?
>> >> >> > vie
>> >> >> > w=log&
>> >> >> > pathrev=205990
>> >> >> >
>> >> >> > ...but not sure this is what I am looking for, and whether there
>> >> >> > is something less drastic than I can do to plug that leak.
>> >> >> >
>> >> >> > Any clue would be very much appreciated.
>> >> >> >
>> >> >> > Thanks.
>> >> >> >
>> >> >> > Sergei
>> >> >> >
>> >> >> > ---
>> >> >> > Qualcomm Innovation Center, Inc. is a member of Code Aurora
>> >> >> > Forum, hosted by The Linux Foundation
>> >> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > LLVM Developers mailing list
>> >> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> >> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> >
>