[LLVMdev] RFC - Extending ProfileInfo to support external profiles

[Chandler Carruth]

On Fri, Sep 6, 2013 at 6:00 AM, Diego Novillo <dnovillo at google.com> wrote:

> Is my understanding more or less correct?
>

Probably for all I know. I'm not aware of anyone actively maintaining or
keeping these pieces working. AFAICT, there have been 3-4 projects that
started hacking on this and didn't finish, and frankly the result is a
mess. It's not clear than any of it really works today.


>  If so, I would like to
> make three main changes:
>
> 1- Unify profile loading so that it only talks to the basic
>    analysis API. Here, I would like to change the way that
>    profile information is reflected in the program. Currently,
>    the compiler loads data from the profile file into 4 arrays
>    inside ProfileInfo.  This is then reflected into the CFG for
>    the corresponding function.
>
>    Instead, I would like to reflect profile info as metadata in
>    the IR. For instance, functions seldom called in the profile
>    could be annotated with the 'cold' attribute in the IR. Block
>    frequency is more straightforward: we add the frequency data
>    to the first instruction of the block.
>
>    Things like that. The problem here is that different profiling
>    mechanisms will have different kinds of data. We will need
>    ways of translating those into metadata that is then served
>    from the analysis API. For example, how do we deal with value
>    profilers?
>
>    For now, I'm focusing on the most common types of profilers.
>    These tend to use block/edge frequencies, which are more
>    straightforward to convert.
>

I really think you should ignore the existing profile support in LLVM. ALl
of it. I actually think we should remove all of it, but that can happen
later.

I would build whatever passes and infrastructure you need in order to load
your profile data, in whatever format makes sense, and annotate the IR with
metadata. I would just build them from scratch. I don't think there is
anything to be gained from trying toclean up the existing profiling
infrastructure when you'll end up re-using almost none of it, you will have
different constraints when dealing with profiles from sampling tools, and
will still have a common IR-level interface in the metadata.


>
> 2- Passes should not need to be aware that profile
>    information exists. They would make the usual calls to the
>    analysis API, which will use profile data, if available.
>
>    With this, I want to make optimizers automatically make
>    better decisions when profile information exists.
>

I believe this has already been done. I'm not aware of any optimization
passes in use today that will need changes here.


>
> 3- Profile data should be flexible wrt code drift. This is
>    something that the profile loader should deal with. It
>    currently fails when the profile is out of sync with the shape
>    of the IR. In the presence of stale profile:
>    (a) Do not fail hard.
>    (b) Make conservative decisions with the annotations it
>        generates.
>    (c) Provide feedback to the user about the staleness of the
>        profile (e.g. % of samples/records dropped from the profile).
>

Yes, in designing your own (from the ground up) loader, these will be
important issues.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130906/6fee487e/attachment.html>