<div dir="ltr"><div><div>This sounds like a problem best solved by tracking source code movement via your source control system.<br></div>If you know the commit of the code that produced the sample, you should be able to use source control history / diffs to translate absolute line numbers to the location where the source has moved.<br></div>This would have the added advantage of highlighting where those samples are likely to be useless or completely misleading.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Oct 25, 2014 at 8:57 AM, Diego Novillo <span dir="ltr"><<a href="mailto:dnovillo@google.com" target="_blank">dnovillo@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br><br><div class="gmail_quote"><span class="">On Fri Oct 24 2014 at 6:21:14 PM David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Oct 24, 2014 at 3:16 PM, Diego Novillo <span dir="ltr"><<a href="mailto:dnovillo@google.com" target="_blank">dnovillo@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br><br><div class="gmail_quote"><span>On Fri Oct 24 2014 at 6:11:21 PM David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Oct 24, 2014 at 2:48 PM, Diego Novillo <span dir="ltr"><<a href="mailto:dnovillo@google.com" target="_blank">dnovillo@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">I'm not sure if this was intended, but it's going to be a problem for sample profiles.<div><br></div><div>When we compile with -gmlt, the profiler expects to find the line number for all the function headers so that it can compute relative line locations for the profile.</div><div><br></div><div>The tool that reads the ELF binary is not finding them, so it writes out absolute line numbers, which are impossible to match during the profile-use phase.</div><div><br></div><div>The problem seems to be that we are missing DW_TAG_subprogram for all the functions in the file.</div><div><br></div><div>Attached are the dwarf dumps of the same program. One compiled with my system's clang 3.4 and the other with today's trunk. In both compiles, I used -gline-tables-only.</div><div><br></div><div>The trunk version is missing all the subprogram tags for the functions in the file. This breaks the sample profiler.</div><div><br></div><div>Should I file a bug, or is -gmlt going to be like this from now on? The latter would be a problem for us.</div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br>Open to negotiation, but this change is intentional ( for details, see the commit: <a href="http://llvm.org/viewvc/llvm-project?rev=218129&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=218129&view=rev</a> ).<br></div></div></div></div></blockquote><div><br></div></span><div>Well, this breaks us. Very hard. It absolutely blocks us from using SamplePGO with LLVM.</div></div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br>Sorry about that - I knew ASan needed gmlt data of a certain form and worked carefully to ensure llvm-symbolizer could still symbolize with these changes, but wasn't aware of this particular dependence from PGO (just assumed it used the line table directly).<br></div></div></div></div></blockquote><div><br></div></span><div>It does, but it uses relative line numbers. That's why it needs the source locs for the function headers.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> </div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div>The alternative would be to make the compiler use absolute line numbers, but in the experience we've collected with GCC, this makes the profiles very brittle wrt source changes.<br></div></div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br>It'd be interesting to see the data - I guess you throw out profiles that don't match on a per-function basis? So adding a few lines to one function will invalidate the profile for that function, but not all the subsequent ones in the file?<br></div></div></div></div></blockquote><div><br></div></span><div>Right. Dehao started using absolute numbers, but then moved to relative numbers when he saw that the performance drops were fairly pronounced as the profile aged. I don't recall the exact drops he noticed.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> </div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div></div><div>I don't have a better idea atm. Would there be any other way to generate relative line numbers?</div></div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br>Possibly.<br> </div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div> Perhaps we could use the first line number we find with samples? </div></div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br>Probably - I guess you use the ELF symbol table to compute the address range of each function? Given that, you could find the smallest line number in that address range in the line table, and make everything else relative to that.<br></div></div></div></div></blockquote><div><br></div></span><div>That could probably work, but I'm not sure how the ELF reading code in the conversion tool does this calculation. I'll check it out.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div> </div></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><div> The problem here is that this line will shift, depending on how the profile was obtained.<br></div></div></blockquote></div></div></div><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><br>Not sure what you're referring to here ^<br></div></div></div></div></blockquote><div><br></div></span><div>Say the function starts at line 20. If in one profile run, we get samples at line 20 and line 23, then we'll compute the relative locations as 0 and 3. But if the first sample you get is at line 21, then you'll compute the relative locations as 0 and 2.</div><div><br></div><div>Using the address ranges in the line table may be the way to go. I'll look at this next week.</div><span class="HOEnZb"><font color="#888888"><div><br></div><div><br></div><div>Diego. </div></font></span></div>
<br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
<br></blockquote></div><br></div>