[PATCH] D147456: [AutoFDO] Stale profile matching(part 1)

[Wenlei He via Phabricator via llvm-commits]

wenlei added a comment.

In D147456#4247035 <https://reviews.llvm.org/D147456#4247035>, @davidxl wrote:

> In D147456#4247011 <https://reviews.llvm.org/D147456#4247011>, @wenlei wrote:
>
>> In D147456#4246994 <https://reviews.llvm.org/D147456#4246994>, @davidxl wrote:
>>
>>> In D147456#4246802 <https://reviews.llvm.org/D147456#4246802>, @wenlei wrote:
>>>
>>>> In D147456#4246749 <https://reviews.llvm.org/D147456#4246749>, @davidxl wrote:
>>>>
>>>>> In D147456#4246695 <https://reviews.llvm.org/D147456#4246695>, @hoy wrote:
>>>>>
>>>>>> In D147456#4246648 <https://reviews.llvm.org/D147456#4246648>, @davidxl wrote:
>>>>>>
>>>>>>> 
>>>>>>
>>>>>>
>>>>>>
>>>>>>> By the way, another way is to offline remapping --- basically 'correcting' the stale profile by mapping the old locations to the new locations and produce a refreshed profile.  This is what we plan to do internally.
>>>>>>
>>>>>> This sounds interesting. By offline do you mean the profile generation time? Wondering how to get the new locations without building the new binary.
>>>>>
>>>>> Right -- the source revision information is available for any target in the build environment.
>>>>
>>>> By doing the mapping during profile generation outside of compiler, you would need a new profile for each target source version. That seems less flexible comparing to doing the mapping in the compiler where the mapping is naturally built for the current source being compiled.
>>>
>>> In practice, it does not need to be -- aka many kinds of source changed don't need to trigger profile updating. Changes to cold source paths don't matter either.  It is ok for the target source revision of the updated profile to be behind that of the actual build.
>>
>> Sure, then you need to make sure the source version you are generating profile for doesn't diverge too much from the source version that you will actually compile. Often times we also can't predict the source version that will actually be used for compilation later, what if a last minute sizable hotfix is landed? What you described should be doable, but more work/complexity IMO. Isn't it easier if this is all handled at compile time transparently within the compiler, given the cost of doing that isn't really measurable.
>
> The updating can be done on-demand based on triggers (source changes or direct user directive). Doing this offline can open up more expensive fuzzy algorithms (open ended).  Having a layer in the compiler to do this is fine -- though would like to see how effective it is for our internal workloads).

Ok. Full disclosure though, we started with ideas around more sophisticated fuzzy graph matching for CSSPGO (we even emit cfg info into profile for some experiments), but settled on simpler algorithm since that worked good enough. We will upstream additional "breaking change" for probe Id assignment later that pushes this to be more effective for CSSPGO.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D147456/new/

https://reviews.llvm.org/D147456