[llvm-dev] Memory utilization problems in profile reader

[Xinliang David Li via llvm-dev]

Can you extract the relevant part of the heap profile data?   How large is
the sample profile data fed to the compiler?

The indexed format profile size for clang is <100MB.  The InstrProfRecord
for each function is read, used and discarded one at a time, so there
should not be problem  as described.

David



On Wed, Dec 9, 2015 at 7:52 AM, Diego Novillo via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
> I've been experimenting with profiled bootstraps using sample profiles.
> Initially, I made stage2 build stage3 while running under Perf.  This
> produced a 20Gb profile which took too long to convert to LLVM, and used
> ~30Gb of RAM.  So, I decided that this was not going to be very useful for
> general usage.
>
> I then changed the bootstrap to instead run each individual compile under
> Perf.  This produced ~2,200 profiles, each of which took up to 1 minute to
> convert, and then they all have to be merged into a single profile.  Also
> didn't like it.
>
> Since all compiles are more or less the same in terms of what the compiler
> does, I decided to take the top 10 biggest profiles and merge those.  That
> seemed to work.  This resulted in a 21Mb profile that I could use as input
> to -fprofile-sample-use.
>
> I started stage 3 of the bootstrap and left it to work.  I noticed it was
> slow, so I thought "we'll need to speed things up".  The build never
> finished.  Instead, ninja crashed my machine.
>
> It turns out that each clang invocation was growing to 4Gb of RSS.  All
> that memory is being allocated by the profile reader (
> https://drive.google.com/file/d/0B9lq1VKvmXKFQVp1cGtZM2RSdWc/view?usp=sharing
> ).
>
> So, heads up, we need to trim it down.  Perhaps by only loading one
> function profile at a time, use it and actively discard it.  Or simply be
> better at flushing the reader data structures as they're used during
> annotations.  I'll be sending patches about this in the coming days.
>
> It's likely that the sample reader is doing something silly here.  Duncan,
> Justin, do you have memories of issues like this one with instrumentation?
> I'll be trying a similar experiment with it after I'm done with the biggest
> issues in the sampler.
>
>
> Thanks.  Diego.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151209/3d062f80/attachment.html>