[LLVMdev] Recent changes in -gmlt break sample profiling

Sun Oct 26 20:56:36 PDT 2014

Figured it had been. Might still be interesting, but I can see where the
function level relative offset is a decent compromise.

-eric

On Sun Oct 26 2014 at 7:31:19 PM Xinliang David Li <xinliangli at gmail.com>
wrote:

> Alternatives like this were discussed during autofdo's original design.
> For instance fetching the source diff and establish line mapping between
> two revisions. Another one is to use more fine grained offsets in profile
> to have better source change tolerance.  We ended up with function level
> relative offset solution which is the simplest and works fine in practice.
> (It is not designed to handle large scale refactoring).
>
> David
>
> On Sun, Oct 26, 2014 at 4:25 PM, Eric Christopher <echristo at gmail.com>
> wrote:
>
>> On Sun, Oct 26, 2014 at 3:47 PM, Jeremy Lakeman
>> <Jeremy.Lakeman at gmail.com> wrote:
>> > This sounds like a problem best solved by tracking source code movement
>> via
>> > your source control system.
>> > If you know the commit of the code that produced the sample, you should
>> be
>> > able to use source control history / diffs to translate absolute line
>> > numbers to the location where the source has moved.
>> > This would have the added advantage of highlighting where those samples
>> are
>> > likely to be useless or completely misleading.
>>
>> This particular scheme might not work, but I was thinking of something
>> similar last week.
>>
>> Basically you'd use the same algorithms that the source control system
>> uses when merging or doing whitespace only diffs etc to construct a
>> better correspondence between source lines. The problem then comes
>> down to "we added some hot code that doesn't have any samples" at
>> which point you'd want the optimizer to pay attention to propagating
>> weights along edges without them.
>>
>> Anyhow, some food for thought.
>>
>> -eric
>>
>> >
>> > On Sat, Oct 25, 2014 at 8:57 AM, Diego Novillo <dnovillo at google.com>
>> wrote:
>> >>
>> >>
>> >>
>> >> On Fri Oct 24 2014 at 6:21:14 PM David Blaikie <dblaikie at gmail.com>
>> wrote:
>> >>>
>> >>> On Fri, Oct 24, 2014 at 3:16 PM, Diego Novillo <dnovillo at google.com>
>> >>> wrote:
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Fri Oct 24 2014 at 6:11:21 PM David Blaikie <dblaikie at gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> On Fri, Oct 24, 2014 at 2:48 PM, Diego Novillo <dnovillo at google.com
>> >
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> I'm not sure if this was intended, but it's going to be a problem
>> for
>> >>>>>> sample profiles.
>> >>>>>>
>> >>>>>> When we compile with -gmlt, the profiler expects to find the line
>> >>>>>> number for all the function headers so that it can compute
>> relative line
>> >>>>>> locations for the profile.
>> >>>>>>
>> >>>>>> The tool that reads the ELF binary is not finding them, so it
>> writes
>> >>>>>> out absolute line numbers, which are impossible to match during the
>> >>>>>> profile-use phase.
>> >>>>>>
>> >>>>>> The problem seems to be that we are missing DW_TAG_subprogram for
>> all
>> >>>>>> the functions in the file.
>> >>>>>>
>> >>>>>> Attached are the dwarf dumps of the same program. One compiled
>> with my
>> >>>>>> system's clang 3.4 and the other with today's trunk. In both
>> compiles, I
>> >>>>>> used -gline-tables-only.
>> >>>>>>
>> >>>>>> The trunk version is missing all the subprogram tags for the
>> functions
>> >>>>>> in the file. This breaks the sample profiler.
>> >>>>>>
>> >>>>>> Should I file a bug, or is -gmlt going to be like this from now on?
>> >>>>>> The latter would be a problem for us.
>> >>>>>
>> >>>>>
>> >>>>> Open to negotiation, but this change is intentional ( for details,
>> see
>> >>>>> the commit: http://llvm.org/viewvc/llvm-project?rev=218129&view=rev
>> ).
>> >>>>
>> >>>>
>> >>>> Well, this breaks us. Very hard. It absolutely blocks us from using
>> >>>> SamplePGO with LLVM.
>> >>>
>> >>>
>> >>> Sorry about that - I knew ASan needed gmlt data of a certain form and
>> >>> worked carefully to ensure llvm-symbolizer could still symbolize with
>> these
>> >>> changes, but wasn't aware of this particular dependence from PGO (just
>> >>> assumed it used the line table directly).
>> >>
>> >>
>> >> It does, but it uses relative line numbers. That's why it needs the
>> source
>> >> locs for the function headers.
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> The alternative would be to make the compiler use absolute line
>> numbers,
>> >>>> but in the experience we've collected with GCC, this makes the
>> profiles very
>> >>>> brittle wrt source changes.
>> >>>
>> >>>
>> >>> It'd be interesting to see the data - I guess you throw out profiles
>> that
>> >>> don't match on a per-function basis? So adding a few lines to one
>> function
>> >>> will invalidate the profile for that function, but not all the
>> subsequent
>> >>> ones in the file?
>> >>
>> >>
>> >> Right. Dehao started using absolute numbers, but then moved to relative
>> >> numbers when he saw that the performance drops were fairly pronounced
>> as the
>> >> profile aged. I don't recall the exact drops he noticed.
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> I don't have a better idea atm. Would there be any other way to
>> generate
>> >>>> relative line numbers?
>> >>>
>> >>>
>> >>> Possibly.
>> >>>
>> >>>>
>> >>>>   Perhaps we could use the first line number we find with samples?
>> >>>
>> >>>
>> >>> Probably - I guess you use the ELF symbol table to compute the address
>> >>> range of each function? Given that, you could find the smallest line
>> number
>> >>> in that address range in the line table, and make everything else
>> relative
>> >>> to that.
>> >>
>> >>
>> >> That could probably work, but I'm not sure how the ELF reading code in
>> the
>> >> conversion tool does this calculation. I'll check it out.
>> >>
>> >>>
>> >>>
>> >>>>
>> >>>> The problem here is that this line will shift, depending on how the
>> >>>> profile was obtained.
>> >>>
>> >>>
>> >>> Not sure what you're referring to here ^
>> >>
>> >>
>> >> Say the function starts at line 20. If in one profile run, we get
>> samples
>> >> at line 20 and line 23, then we'll compute the relative locations as 0
>> and
>> >> 3. But if the first sample you get is at line 21, then you'll compute
>> the
>> >> relative locations as  0 and 2.
>> >>
>> >> Using the address ranges in the line table may be the way to go. I'll
>> look
>> >> at this next week.
>> >>
>> >>
>> >> Diego.
>> >>
>> >> _______________________________________________
>> >> LLVM Developers mailing list
>> >> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> >>
>> >
>> >
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> >
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141027/5034e90d/attachment.html>