[llvm-dev] Profiling data structure

[Roger Pau Monné via llvm-dev]

On Thu, Oct 26, 2017 at 11:07:51AM -0700, Xinliang David Li wrote:
> On Thu, Oct 26, 2017 at 1:31 AM, Roger Pau Monné <roger.pau at citrix.com>
> wrote:
> 
> > On Wed, Oct 25, 2017 at 09:13:54AM -0700, Xinliang David Li wrote:
> > > On Wed, Oct 25, 2017 at 12:26 AM, Roger Pau Monné via llvm-dev <
> > > >  - The "Values" field in __llvm_profile_data always seems to be NULL.
> > > >    Is this expected? What/why could cause this?
> > > >
> > >
> > > This is for value profiling. For now there are only two kinds:  indirect
> > > call targets and memcpy/memset size. If the function body does not have
> > any
> > > value sites, the field will be null.
> >
> > You will have to bear with me because my knowledge of compiler
> > internals is very limited. What is exactly a "value site"?
> >
> >
> It refers to a location in a function where the compiler inserts the value
> profiling hook.
> 
> 
> 
> > Can you give me some code examples that trigger this in C?
> >
> 
> 
> typedef void (*FP)(void);
> 
> void test (FP fp) {
> 
>      fp();        /* a value site */
> }
> 
> 
> There are two ways to turn on value profiling:
> 
> 1) using instrumentation for PGO (we call it IR-PGO):
> 
>    -fprofile-generate
> 
> 
> 2) using Front-end based instrumentation which is used for coverage testing:
> 
>    -fprofile-instr-generate -mllvm -enable-value-profiling=true

I'm currently using -fprofile-instr-generate -fcoverage-mapping which
are the options specified in [0]. ATM I'm only interested in getting
coverage data.

I've created the following simple example:

static void foo(void)
{
  int foo = 1;
}

static void bar(void)
{
  int bar = 1;
}

static void exec(void (*fp)(void))
{
  fp();
}

int main(int argc, char **argv)
{
  int count = 0;

  while ( count++ < 10000 )
    exec(rand() % 2 ? foo : bar);

  return 0;
}

Then I've used my custom made runtime and checked that both Values and
NumValueSites inside of __llvm_profile_data are still NULL.

I've also dumped the coverage data using my own runtime (which ignores
Values and NumValueSites), and got the following profile, which looks
right:

     |static void foo(void)
4.95k|{
4.95k|  int foo = 1;
4.95k|}
     |
     |static void bar(void)
5.04k|{
5.04k|  int bar = 1;
5.04k|}
     |
     |static void exec(void (*fp)(void))
10.0k|{
10.0k|  fp();
10.0k|}
     |
     |int main(int argc, char **argv)
    1|{
    1|  int count = 0;
    1|
10.0k|  while ( count++ < 10000 )
10.0k|    exec(rand() % 2 ? foo : bar);
    1|
    1|  return 0;
    1|}

So can Values and NumValueSites be safely ignored in order to obtain
the coverage data?

> >
> > > >
> > > >  - Since what I'm coding is a decoupled replacement for the profiling
> > > >    runtime inside of compiler_rt, is there anyway that at compile or
> > > >    run time I can fetch the version of the profiling data?
> > > >
> > >
> > >  At compile time, it is the macro: INST_PROF_RAW_VERSION. At runtime, it
> > is
> > > the second field of the raw profile header.
> >
> > I'm not able to use INST_PROF_RAW_VERSION at compile time. Are you
> > sure this is exported? If I do:
> >
> > cc -fprofile-instr-generate -fcoverage-mapping -dM -E - < /dev/null
> >
> >
> Ok.  The macro is defined for building the compiler itself, but not passed
> down to the user program when compiling it.
> 
> If you use IR-PGO (turned on with -fprofile-generate), the information is
> stored in a symbol in rodata: __llvm_profile_raw_version -- the least
> significant 32bits has the value of the raw version. Unfortunately, the
> front-end based instrumentation currently does not emit such a symbol.

That's right, __llvm_profile_raw_version cannot be used in my case
because it's exported by the runtime, and here I'm not using the
compiler_rt runtime at all.

Would it be possible to export this as a compile time define when
-fprofile-instr-generate -fcoverage-mapping is used?

Does that sound sensible?

> > I don't see INST_PROF_RAW_VERSION neither any similar defines.
> >
> > >    I'm mostly worried that at some point llvm will bump the version
> > > >    and change the layout, and then I will have to update my runtime
> > > >    accordingly, but without llvm reporting the version used by the
> > > >    compiler it's going to be very hard to keep backwards
> > > >    compatibility or to detect version bumps.
> > > >
> > > >
> > > yes, the raw profile format can change anytime. We only try to keep
> > > backward compatibility for indexed format.
> > >
> > > At runtime with in-process profile merging, if the source raw profile
> > > data's format version is different from the current runtime version, an
> > > error will be emitted.
> >
> > Keep in mind this is a kernel, so the source is compiled with
> > "-fprofile-instr-generate -fcoverage-mapping", but the profiling
> > runtime in compiler_rt is not linked against the kernel.
> >
> > I would like to have a reliable way that I could use to detect version
> > bumps in the internal coverage data, so that I can implement the
> > required changes in my in-kernel coverage code.
> >
> > I have a series ready for Xen in order to implement this, I will send
> > the patch with the in-kernel profiling implementation to this list for
> > review.
> >
> 
> 
> The right way for this is to define __llvm_profile_raw_version variable
> with FE instrumentation as is done by IR-PGO.   I have cc'ed Vedant who may
> help with this.

Wouldn't it be better to export an internal compiler define rather
than creating a symbol with the profile version?

Thanks, Roger.

[0] https://clang.llvm.org/docs/SourceBasedCodeCoverage.html