[llvm-dev] Memory utilization problems in profile reader

[Diego Novillo via llvm-dev]

So, I traced it down to the DenseMaps in class FunctionSamples.  I've
replaced them with two std::vector, and the read operation causes the
compiler to grow from 70Mb to 280Mb.  With the DenseMaps, reading the
profile causes the compiler to grow from 70Mb to 3Gb.

Somehow the DenseMaps are causing a 10x growth factor.  Those keys are
probably an issue.  Or perhaps we just need a different representation for
sample records and call sites.

On Thu, Dec 10, 2015 at 3:56 PM, Diego Novillo <dnovillo at google.com> wrote:

>
>
> On Wed, Dec 9, 2015 at 3:14 PM, Xinliang David Li <xinliangli at gmail.com>
> wrote:
>
>> Can you extract the relevant part of the heap profile data?
>>
>
> It's all profile data, actually.  The heap utilization is massively
> dominated by the profile reader.
>
>
>
>>   How large is the sample profile data fed to the compiler?
>>
>>
> For this run, the input file was 21Mb.
>
>
>> The indexed format profile size for clang is <100MB.  The InstrProfRecord
>> for each function is read, used and discarded one at a time, so there
>> should not be problem  as described.
>>
>
> Good.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151211/e44831b1/attachment.html>