<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 27, 2017 at 3:28 AM, Roger Pau Monné <span dir="ltr"><<a href="mailto:roger.pau@citrix.com" target="_blank">roger.pau@citrix.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-">On Thu, Oct 26, 2017 at 11:07:51AM -0700, Xinliang David Li wrote:<br>
> On Thu, Oct 26, 2017 at 1:31 AM, Roger Pau Monné <<a href="mailto:roger.pau@citrix.com">roger.pau@citrix.com</a>><br>
> wrote:<br>
><br>
> > On Wed, Oct 25, 2017 at 09:13:54AM -0700, Xinliang David Li wrote:<br>
> > > On Wed, Oct 25, 2017 at 12:26 AM, Roger Pau Monné via llvm-dev <<br>
</span><span class="gmail-">> > > > - The "Values" field in __llvm_profile_data always seems to be NULL.<br>
> > > > Is this expected? What/why could cause this?<br>
> > > ><br>
> > ><br>
> > > This is for value profiling. For now there are only two kinds: indirect<br>
> > > call targets and memcpy/memset size. If the function body does not have<br>
> > any<br>
> > > value sites, the field will be null.<br>
> ><br>
> > You will have to bear with me because my knowledge of compiler<br>
> > internals is very limited. What is exactly a "value site"?<br>
> ><br>
> ><br>
> It refers to a location in a function where the compiler inserts the value<br>
> profiling hook.<br>
><br>
><br>
><br>
> > Can you give me some code examples that trigger this in C?<br>
> ><br>
><br>
><br>
> typedef void (*FP)(void);<br>
><br>
> void test (FP fp) {<br>
><br>
> fp(); /* a value site */<br>
> }<br>
><br>
><br>
> There are two ways to turn on value profiling:<br>
><br>
> 1) using instrumentation for PGO (we call it IR-PGO):<br>
><br>
> -fprofile-generate<br>
><br>
><br>
> 2) using Front-end based instrumentation which is used for coverage testing:<br>
><br>
> -fprofile-instr-generate -mllvm -enable-value-profiling=true<br>
<br>
</span>I'm currently using -fprofile-instr-generate -fcoverage-mapping which<br>
are the options specified in [0]. ATM I'm only interested in getting<br>
coverage data.<br>
<br>
I've created the following simple example:<br>
<br>
static void foo(void)<br>
{<br>
int foo = 1;<br>
}<br>
<br>
static void bar(void)<br>
{<br>
int bar = 1;<br>
}<br>
<br>
static void exec(void (*fp)(void))<br>
{<br>
fp();<br>
}<br>
<br>
int main(int argc, char **argv)<br>
{<br>
int count = 0;<br>
<br>
while ( count++ < 10000 )<br>
exec(rand() % 2 ? foo : bar);<br>
<br>
return 0;<br>
}<br>
<br>
Then I've used my custom made runtime and checked that both Values and<br>
NumValueSites inside of __llvm_profile_data are still NULL.<br></blockquote><div><br></div><div><br></div><div>It should not. The IR dump shows:</div><div><br></div><div><font face="monospace, monospace"><br></font></div><div><div><font face="monospace, monospace">@__profd_ic.c_exec = private global { i64, i64, i64*, i8*, i8*, i32, [2 x i16] } { i64 3252020653354712924, i64 0, i64* getelementptr inbounds ([1 x i64], [1 x i64]* @__profc_ic.c_exec, i32 0, i32 0), i8* null, i8* bitcast ([1 x i64]* @__profvp_ic.c_exec to i8*), i32 1, [2 x i16] [i16 1, i16 0] }, section "__llvm_prf_data", align 8</font></div></div><div><font face="monospace"> </font><br></div><div><font face="monospace, monospace">The Value's field is </font><font face="monospace"> i8* bitcast ([1 x i64]* @__profvp_ic.c_exec to i8*), and num of value sites for the kind is 1.</font></div><div><font face="monospace, monospace"> </font></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
I've also dumped the coverage data using my own runtime (which ignores<br>
Values and NumValueSites), and got the following profile, which looks<br>
right:<br>
<br>
|static void foo(void)<br>
4.95k|{<br>
4.95k| int foo = 1;<br>
4.95k|}<br>
|<br>
|static void bar(void)<br>
5.04k|{<br>
5.04k| int bar = 1;<br>
5.04k|}<br>
|<br>
|static void exec(void (*fp)(void))<br>
10.0k|{<br>
10.0k| fp();<br>
10.0k|}<br>
|<br>
|int main(int argc, char **argv)<br>
1|{<br>
1| int count = 0;<br>
1|<br>
10.0k| while ( count++ < 10000 )<br>
10.0k| exec(rand() % 2 ? foo : bar);<br>
1|<br>
1| return 0;<br>
1|}<br>
<br>
So can Values and NumValueSites be safely ignored in order to obtain<br>
the coverage data?<br></blockquote><div><br></div><div>Yes, value profiles are not used for coverage testing, it is for PGO only.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<span class="gmail-"><br>
> ><br>
> > > ><br>
> > > > - Since what I'm coding is a decoupled replacement for the profiling<br>
> > > > runtime inside of compiler_rt, is there anyway that at compile or<br>
> > > > run time I can fetch the version of the profiling data?<br>
> > > ><br>
> > ><br>
> > > At compile time, it is the macro: INST_PROF_RAW_VERSION. At runtime, it<br>
> > is<br>
> > > the second field of the raw profile header.<br>
> ><br>
> > I'm not able to use INST_PROF_RAW_VERSION at compile time. Are you<br>
> > sure this is exported? If I do:<br>
> ><br>
> > cc -fprofile-instr-generate -fcoverage-mapping -dM -E - < /dev/null<br>
> ><br>
> ><br>
> Ok. The macro is defined for building the compiler itself, but not passed<br>
> down to the user program when compiling it.<br>
><br>
> If you use IR-PGO (turned on with -fprofile-generate), the information is<br>
> stored in a symbol in rodata: __llvm_profile_raw_version -- the least<br>
> significant 32bits has the value of the raw version. Unfortunately, the<br>
> front-end based instrumentation currently does not emit such a symbol.<br>
<br>
</span>That's right, __llvm_profile_raw_version cannot be used in my case<br>
because it's exported by the runtime, and here I'm not using the<br>
compiler_rt runtime at all.<br>
<br></blockquote><div><br></div><div>Not really. The compile_rt's version is a weak symbol. The one defined by the compiler is the strong definition. IR PGO always defines it so there is no dependency on runtime. However the current problem is that frontend instrumentation does not define this symbol.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Would it be possible to export this as a compile time define when<br>
-fprofile-instr-generate -fcoverage-mapping is used?<br>
<br></blockquote><div><br></div><div>It can, but the use case seems too narrow to be generally useful.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Does that sound sensible?<br>
<span class="gmail-"><br>
> > I don't see INST_PROF_RAW_VERSION neither any similar defines.<br>
> ><br>
> > > I'm mostly worried that at some point llvm will bump the version<br>
> > > > and change the layout, and then I will have to update my runtime<br>
> > > > accordingly, but without llvm reporting the version used by the<br>
> > > > compiler it's going to be very hard to keep backwards<br>
> > > > compatibility or to detect version bumps.<br>
> > > ><br>
> > > ><br>
> > > yes, the raw profile format can change anytime. We only try to keep<br>
> > > backward compatibility for indexed format.<br>
> > ><br>
> > > At runtime with in-process profile merging, if the source raw profile<br>
> > > data's format version is different from the current runtime version, an<br>
> > > error will be emitted.<br>
> ><br>
> > Keep in mind this is a kernel, so the source is compiled with<br>
> > "-fprofile-instr-generate -fcoverage-mapping", but the profiling<br>
> > runtime in compiler_rt is not linked against the kernel.<br>
> ><br>
> > I would like to have a reliable way that I could use to detect version<br>
> > bumps in the internal coverage data, so that I can implement the<br>
> > required changes in my in-kernel coverage code.<br>
> ><br>
> > I have a series ready for Xen in order to implement this, I will send<br>
> > the patch with the in-kernel profiling implementation to this list for<br>
> > review.<br>
> ><br>
><br>
><br>
> The right way for this is to define __llvm_profile_raw_version variable<br>
> with FE instrumentation as is done by IR-PGO. I have cc'ed Vedant who may<br>
> help with this.<br>
<br>
</span>Wouldn't it be better to export an internal compiler define rather<br>
than creating a symbol with the profile version?<br>
<br></blockquote><div><br></div><div>It is certainly doable, but I don't see any existing macros that are similar to this use case. You may want to bring this up to cfe-dev.</div><div><br></div><div>thanks,</div><div><br></div><div>David</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Thanks, Roger.<br>
<br>
[0] <a href="https://clang.llvm.org/docs/SourceBasedCodeCoverage.html" rel="noreferrer" target="_blank">https://clang.llvm.org/docs/<wbr>SourceBasedCodeCoverage.html</a><br>
</blockquote></div><br></div></div>