[llvm-dev] RFC: EfficiencySanitizer working set tool

[Qin Zhao via llvm-dev]

On Wed, Apr 20, 2016 at 1:15 PM, Filipe Cabecinhas <filcab at gmail.com> wrote:

> On Wed, Apr 20, 2016 at 6:44 AM, Derek Bruening via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> > We will divide the program execution into regular intervals and record a
> > snapshot of the working set at each interval boundary.  An adaptive
> strategy
> > can keep the number of snapshots bounded across varying total execution
> time
> > by combining adjacent snapshots via logical or.  When a snapshot is
> > recorded, the shadow memory is cleared so that the next snapshot starts
> with
> > a blank slate.  If we use a 64 byte to 1 byte shadow mapping, we can use
> the
> > upper bits to store up to 8 consecutive snapshots in the shadow memory
> > itself by shifting rather than clearing the shadow memory on a snapshot.
> I forgot to add my previous questions:
>
> About the working set tool:
> How are you thinking about doing the snapshots? How do you plan to
> sync the several threads?
> Spawning an external process/"thread" (kind of like LSan), or internally?
>

I do not think synchronized operation is important here.
The tool should be able to tolerate certain level of racy updates.
We could either use the spawning thread or a simple time interrupt without
sideline thread.
Sampling without sideline thread has the advantage of providing some sample
call stack.
Sideline thread avoids app thread pausing for the scanning.
We may implement both.


>
> I know I'm getting to low(ish)-level details, but if you've already
> thought about this (possibly about several ways to do this), I'd like
> to know more about what avenues you're planning on exploring.
>
> I'm ok with a "we haven't thought that far ahead about this tool yet" :-)
>
> Thank you,
>
>   Filipe
>
> > Recording the snapshot will use optimizations to avoid storing for the
> > entire address space: only mapped regions will be saved.
> >
> > The report from the tool to the user at the end of execution would
> > essentially be a histogram with time on the x axis and the cache lines
> > touched on the y axis.  The units of time could be selected by the user:
> cpu
> > time, wall clock time, memory access count, etc.  The unit determines the
> > method of snapshot sampling, whether performance counters for the memory
> > access count (or instrumentation to increment a global counter) or an
> > itimer.  We can record callstacks with each snapshot as well to help
> give an
> > indication of what the program is doing at that point in time, to help
> the
> > user understand the program phases.
> >
> > We expect the time overhead of the tool to be well under the 5x
> > EfficiencySanitizer ceiling; presumably it should be under 3x.
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "efficiency-sanitizer" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to efficiency-sanitizer+unsubscribe at google.com.
> To post to this group, send email to efficiency-sanitizer at google.com.
> To view this discussion on the web visit
> https://groups.google.com/a/google.com/d/msgid/efficiency-sanitizer/CAEDE853L4qXUw7YwwYWYLkyRRc6caT%3DgiW8%2BFS8%3D6fg6sT5ufg%40mail.gmail.com
> .
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160420/eac592ac/attachment.html>