[LLVMdev] Looking for a fix to memory leak in DWARF support

[Eric Christopher]

On Fri, Jun 13, 2014 at 2:28 PM, Sergei Larin <slarin at codeaurora.org> wrote:
>
> Thanks Eric,
>
>
> They are doing LTO build but with some custom modifications (think a library at a time as opposed to a whole program). I must admit, it is a rather large application as well, so as expected, any inefficiencies are multiplied greatly.
>
> From little that I have seen so far, it looks like debug metadata for an IR object linger behind once the object itself is eliminated (optimized). Rings a bell?
>

They're going to run into all sorts of problems. They're doomed if
it's a large C++ application (or really even if it's a C based
application).

They won't get this to link likely with LTO + debug. It's unlikely to
be a memory leak. It's just an explosion of debug info because of our
lack of uniquing for type information. You can see my EuroLLVM 2013
talk where I describe this and Manman's patches to help fix the issue
over the last couple of years.

-eric

> Thanks.
>
> Sergei
>
> ---
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
>
>> -----Original Message-----
>> From: Eric Christopher [mailto:echristo at gmail.com]
>> Sent: Friday, June 13, 2014 4:00 PM
>> To: Sergei Larin
>> Cc: David Blaikie; LLVM Developers Mailing List
>> Subject: Re: [LLVMdev] Looking for a fix to memory leak in DWARF support
>>
>> On Fri, Jun 13, 2014 at 11:03 AM, Sergei Larin <slarin at codeaurora.org> wrote:
>> >
>> > David,
>> >
>> > Thanks for the quick response...
>> >
>> >
>> > No, at this point I am just getting into the issue... I assume it is a leak, but
>> no clear proof yet. I was hoping it was an obvious thing since I recall a
>> discussion about it a while ago... but maybe I am just confused.
>> >
>> > Was your work for compressing DWARF data  motivated by a certain
>> inefficiency in debug info representation? Did it result in significant savings?
>> >
>>
>> The compression isn't related to the IR level and some data is here on that
>> compressing:
>>
>> https://gcc.gnu.org/wiki/DebugFission
>>
>> The internal representation and code have change significantly since then
>> and there's no real way to help much. It may be a memory leak, but I don't
>> recall one that bad back then. How is your customer compiling? LTO?
>> Something else?
>>
>> -eric
>>
>> > Thanks.
>> >
>> > Sergei
>> >
>> > ---
>> > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>> > hosted by The Linux Foundation
>> >
>> >
>> >> -----Original Message-----
>> >> From: David Blaikie [mailto:dblaikie at gmail.com]
>> >> Sent: Friday, June 13, 2014 12:41 PM
>> >> To: Sergei Larin
>> >> Cc: LLVM Developers Mailing List
>> >> Subject: Re: Looking for a fix to memory leak in DWARF support
>> >>
>> >> Are you sure it's a leak, not just a large memory footprint?
>> >>
>> >> A lot of things have changed. Heck, even the bugs that we've fixed
>> >> might not have existed two years ago (I've certainly created (and
>> >> fixed) my share of bugs in debug info).
>> >>
>> >> Best you start with a leak tracking tool and at least be able to
>> >> point to the major leak culprit (eg: "90% of the leaked memory was
>> >> allocated at <stack
>> >> trace>")
>> >>
>> >> On Fri, Jun 13, 2014 at 9:16 AM, Sergei Larin <slarin at codeaurora.org>
>> wrote:
>> >> >
>> >> > David, (and everyone else)
>> >> >
>> >> >   I am forced to do some maintenance work on a fairly old LLVM
>> >> > branch (likely based on release 3.1) that among other issues has a
>> >> > major problem with memory leak somewhere around DWARF debug
>> support.
>> >> > In fact customer is unable to build with -g at all - simply running
>> >> > out of memory on their project...
>> >> >
>> >> >   I seem to remember that there has been a major fix related to it,
>> >> > but finding it hard to pinpoint. So the appeal to the communal
>> >> > memory
>> >> > - do you remember if  there was a single fix (since the middle of
>> >> > 2012 or so) that would have addressed such issue?
>> >> >
>> >> >   I know the code was drastically change since - I have seen this
>> >> > for instance
>> >> > llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/ELFObjectWriter.cpp?
>> >> > vie
>> >> > w=log&
>> >> > pathrev=205990
>> >> >
>> >> > ...but not sure this is what I am looking for, and whether there is
>> >> > something less drastic than I can do to plug that leak.
>> >> >
>> >> > Any clue would be very much appreciated.
>> >> >
>> >> > Thanks.
>> >> >
>> >> > Sergei
>> >> >
>> >> > ---
>> >> > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>> >> > hosted by The Linux Foundation
>> >> >
>> >
>> >
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>