[llvm-dev] Memory utilization problems in profile reader

[Xinliang David Li via llvm-dev]

On Fri, Dec 11, 2015 at 6:58 AM, Diego Novillo <dnovillo at google.com> wrote:

> So, I traced it down to the DenseMaps in class FunctionSamples.  I've
> replaced them with two std::vector, and the read operation causes the
> compiler to grow from 70Mb to 280Mb.  With the DenseMaps, reading the
> profile causes the compiler to grow from 70Mb to 3Gb.
>
>
The growth from 70Mb to 280Mb for a ~20Mb profile is also alarming, IMO.

David


> Somehow the DenseMaps are causing a 10x growth factor.  Those keys are
> probably an issue.  Or perhaps we just need a different representation for
> sample records and call sites.
>
> On Thu, Dec 10, 2015 at 3:56 PM, Diego Novillo <dnovillo at google.com>
> wrote:
>
>>
>>
>> On Wed, Dec 9, 2015 at 3:14 PM, Xinliang David Li <xinliangli at gmail.com>
>> wrote:
>>
>>> Can you extract the relevant part of the heap profile data?
>>>
>>
>> It's all profile data, actually.  The heap utilization is massively
>> dominated by the profile reader.
>>
>>
>>
>>>   How large is the sample profile data fed to the compiler?
>>>
>>>
>> For this run, the input file was 21Mb.
>>
>>
>>> The indexed format profile size for clang is <100MB.  The
>>> InstrProfRecord for each function is read, used and discarded one at a
>>> time, so there should not be problem  as described.
>>>
>>
>> Good.
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151211/fdee4d92/attachment.html>