[PATCH] D20993: [lit] Add support for PGO profile and code coverage collection

[Xinliang David Li via llvm-commits]

On Fri, Jun 3, 2016 at 9:24 PM, Vedant Kumar <vsk at apple.com> wrote:

> vsk added a comment.
>
> In http://reviews.llvm.org/D20993#449092, @MatzeB wrote:
>
> > This adds a bigger bunch of code to lit, I wonder if it is necessary:
> >
> > - I assume the necessity for the logic inside lit is just the fact that
> the profile data overwrites each other after running a command. Is setting
> LLVM_PROFILE_FILE with a '%p' placeholder not enough to avoid this?
>
>
> Using PID substitution gets close to solving the problem of having
> overwritten profiles, but not all the way. On 32-bit systems PID wraparound
> would pose a real problem. In this patch I include the hash of the test
> command to minimize loss of profiles.
>
> > The final profile data merging can always be done outside of llvm-lit
> afterwards.
>
>
> On a practical level, I don't think this is possible. Raw profiles are too
> large. Turning off the cleanup step and running check-llvm produces over
> half a terrabyte of data. There are a few ways to address this without
> touching lit:
>
> 1. Reduce the size of raw profiles. This would make the compiler runtime
> larger and more complex.
> 2. Run a monitor process that does the merging/cleanup. I don't think that
> approach has any advantages compared to modifying lit.
>
> Alternatively, we could do in-place raw profile merging in the compiler
> runtime. I don't think that's preferable because we'd have to introduce a
> lot of complexity to compiler-rt (e.g portable mandatory file locking).
>
>
Portable file locking should not be a concern. Most unix like platforms
support file locking with fcntl. On Windows, createFile API can be used.

The RFC about this a while ago has lots of performance data on various
types of file systems.  There will be other legit use cases for in-process
merging support as well.

David



> > Or are there other reasons why you need to intercept the opt/llc/etc.
> calls?
>
>
> Intercepting the calls makes it possible to generate 'unique' hashes for
> the profiles, see my comment above.
>
> > - The lit tests don't strike me as the most typical llvm usage (they are
> often designed to test corner cases of the compiler), wouldn't it be better
> to gather profiles by compiling the llvm test-suite or something similar?
>
>
> It's true that the profiles gathered from check-llvm aren't ideal for PGO.
> But, they are perfect for measuring code coverage of the llvm codebase.
> Moreover because lit is portable, other projects can use these changes to
> gather coverage reports with their tests.
>
> (IIRC the test-suite also uses lit?)
>
>
> http://reviews.llvm.org/D20993
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160603/98fdc1de/attachment.html>