[llvm-dev] Memory utilization problems in profile reader

[Diego Novillo via llvm-dev]

Yes, but that's a second order issue. I'll deal with that after this one's
settled.
On Dec 11, 2015 12:04 PM, "Xinliang David Li" <xinliangli at gmail.com> wrote:

>
>
> On Fri, Dec 11, 2015 at 6:58 AM, Diego Novillo <dnovillo at google.com>
> wrote:
>
>> So, I traced it down to the DenseMaps in class FunctionSamples.  I've
>> replaced them with two std::vector, and the read operation causes the
>> compiler to grow from 70Mb to 280Mb.  With the DenseMaps, reading the
>> profile causes the compiler to grow from 70Mb to 3Gb.
>>
>>
> The growth from 70Mb to 280Mb for a ~20Mb profile is also alarming, IMO.
>
> David
>
>
>> Somehow the DenseMaps are causing a 10x growth factor.  Those keys are
>> probably an issue.  Or perhaps we just need a different representation for
>> sample records and call sites.
>>
>> On Thu, Dec 10, 2015 at 3:56 PM, Diego Novillo <dnovillo at google.com>
>> wrote:
>>
>>>
>>>
>>> On Wed, Dec 9, 2015 at 3:14 PM, Xinliang David Li <xinliangli at gmail.com>
>>> wrote:
>>>
>>>> Can you extract the relevant part of the heap profile data?
>>>>
>>>
>>> It's all profile data, actually.  The heap utilization is massively
>>> dominated by the profile reader.
>>>
>>>
>>>
>>>>   How large is the sample profile data fed to the compiler?
>>>>
>>>>
>>> For this run, the input file was 21Mb.
>>>
>>>
>>>> The indexed format profile size for clang is <100MB.  The
>>>> InstrProfRecord for each function is read, used and discarded one at a
>>>> time, so there should not be problem  as described.
>>>>
>>>
>>> Good.
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151211/8c44fa95/attachment.html>