[llvm-dev] Profiling data structure

[Xinliang David Li via llvm-dev]

On Thu, Oct 26, 2017 at 1:31 AM, Roger Pau Monné <roger.pau at citrix.com>
wrote:

> On Wed, Oct 25, 2017 at 09:13:54AM -0700, Xinliang David Li wrote:
> > On Wed, Oct 25, 2017 at 12:26 AM, Roger Pau Monné via llvm-dev <
> > llvm-dev at lists.llvm.org> wrote:
> >
> > > Hello,
> > >
> > > I've been working on implementing some basic functionality in order to
> > > use the llvm profiling functionality inside of a kernel (the Xen
> > > hypervisor). The only functionality I'm interested in is being able to
> > > reset the counters, get the size of the data, and dump the data into a
> > > memory buffer.
> > >
> > > I have to admit I haven't been able to find a lot of documentation
> > > about how this data is stored inside of the different sections, but
> > > with the code in compiler_rt/libs/profile I've been able to craft
> > > something that seems to do something sensible.
> > >
> > > I have however a couple of questions:
> > >
> > >  - The "Values" field in __llvm_profile_data always seems to be NULL.
> > >    Is this expected? What/why could cause this?
> > >
> >
> > This is for value profiling. For now there are only two kinds:  indirect
> > call targets and memcpy/memset size. If the function body does not have
> any
> > value sites, the field will be null.
>
> You will have to bear with me because my knowledge of compiler
> internals is very limited. What is exactly a "value site"?
>
>
It refers to a location in a function where the compiler inserts the value
profiling hook.



> Can you give me some code examples that trigger this in C?
>


typedef void (*FP)(void);

void test (FP fp) {

     fp();        /* a value site */
}


There are two ways to turn on value profiling:

1) using instrumentation for PGO (we call it IR-PGO):

   -fprofile-generate


2) using Front-end based instrumentation which is used for coverage testing:

   -fprofile-instr-generate -mllvm -enable-value-profiling=true





>
> > >
> > >  - Since what I'm coding is a decoupled replacement for the profiling
> > >    runtime inside of compiler_rt, is there anyway that at compile or
> > >    run time I can fetch the version of the profiling data?
> > >
> >
> >  At compile time, it is the macro: INST_PROF_RAW_VERSION. At runtime, it
> is
> > the second field of the raw profile header.
>
> I'm not able to use INST_PROF_RAW_VERSION at compile time. Are you
> sure this is exported? If I do:
>
> cc -fprofile-instr-generate -fcoverage-mapping -dM -E - < /dev/null
>
>
Ok.  The macro is defined for building the compiler itself, but not passed
down to the user program when compiling it.

If you use IR-PGO (turned on with -fprofile-generate), the information is
stored in a symbol in rodata: __llvm_profile_raw_version -- the least
significant 32bits has the value of the raw version. Unfortunately, the
front-end based instrumentation currently does not emit such a symbol.



> I don't see INST_PROF_RAW_VERSION neither any similar defines.
>
> >    I'm mostly worried that at some point llvm will bump the version
> > >    and change the layout, and then I will have to update my runtime
> > >    accordingly, but without llvm reporting the version used by the
> > >    compiler it's going to be very hard to keep backwards
> > >    compatibility or to detect version bumps.
> > >
> > >
> > yes, the raw profile format can change anytime. We only try to keep
> > backward compatibility for indexed format.
> >
> > At runtime with in-process profile merging, if the source raw profile
> > data's format version is different from the current runtime version, an
> > error will be emitted.
>
> Keep in mind this is a kernel, so the source is compiled with
> "-fprofile-instr-generate -fcoverage-mapping", but the profiling
> runtime in compiler_rt is not linked against the kernel.
>
> I would like to have a reliable way that I could use to detect version
> bumps in the internal coverage data, so that I can implement the
> required changes in my in-kernel coverage code.
>
> I have a series ready for Xen in order to implement this, I will send
> the patch with the in-kernel profiling implementation to this list for
> review.
>


The right way for this is to define __llvm_profile_raw_version variable
with FE instrumentation as is done by IR-PGO.   I have cc'ed Vedant who may
help with this.

thanks,

David



>
> Thanks, Roger.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171026/6289105f/attachment.html>