[PATCH] D104060: Machine IR Profile

[Kyungwoo Lee via Phabricator via llvm-commits]

kyulee added a comment.

In D104060#2837314 <https://reviews.llvm.org/D104060#2837314>, @davidxl wrote:

> In D104060#2837248 <https://reviews.llvm.org/D104060#2837248>, @ellis wrote:
>
>> In D104060#2836209 <https://reviews.llvm.org/D104060#2836209>, @davidxl wrote:
>>
>>> In D104060#2836015 <https://reviews.llvm.org/D104060#2836015>, @kyulee wrote:
>>>
>>>> Thanks for the patch D104556 <https://reviews.llvm.org/D104556> that uses PC relative relocations for reference! This avoids dynamic relocation, which is the prerequisite to make a section (region) be extractable.
>>>> Unfortunately this is not sufficient because we need to find the references once they are extracted out because the address ranges (either in the original binary and the extracted binary) are not valid or can be altered.
>>>> So, we need some notion of section (or region) relative addressing which is not all available in COFF/ELF/MachO. One trick in MIP is to introduce an anchor reference to express such addressing with two PC relative relocations.
>>>> In fact, this anchor reference can be hoisted out into the header which we can share, which is not in this change yet.
>>>
>>> This is interesting.  Can you describe a little more with the fuzzy matching, perhaps with a small example to demonstrate how it works?
>>
>> If you're asking about our trick for the raw profile addresses, I have a comment above that describes it in detail (the comment title is "Section Layout"). If your asking about something else, could you please elaborate?
>
> I was referring to this comment by kyulee@:  "because the address ranges (either in the original binary and the extracted binary) are not valid or can be altered." --  Is this about using stale profile  or something else? I might have misunderstood the part about 'address ranges can be altered' part.

Say a binary consists of two sections, C and D.
When D is extracted out (and thus C only remains), the extracted binary D starts with 0 while the address of D in the prior (or original) binary used to be after C. So, a direct addressing is simply invalid unless we bookeep the original location somehow.
In addition, we also saw the case where even C's address in the resulting (remaining) binary might be shifted via a strip process that seemed to alter the image base. 
So, I think we need to express and operate with the references which are constant within each section or region.

This is a longer version, and I hope it helps for clarification on how and why MIP works in this context.
Let say we have two sections C (raw counter) and D (metadata) that has references to C where D is to be extracted out (at build-time).

  CHeader:
  C1:
  C2:

Ideally, we want to layout D as below so that the contents in D are constant, but this is not possible in all platforms.

  DHeader:
  D1:
    &C1 - &CHeader // Section/region-relative to C1
  D2:
    &C2 - &CHeader // Section/region-relative to C2

This is how MIP lays D in this patch.
This simulates section-relative expression with two PC-relative expressions. This incurs extra size for D but in practice, this is not a concern because D is meant to be serialized (or extracted) at build-time.
That being said, MIP does not optimize D for size but for simplicity in the maintenance. All metadata including name, etc. is wrapped and self-contained in each D's entry (function granularity) without spanning multiple metadata sections.
Note D is a sort of raw form of MIP's metadata, which is turned into an optimized (merged) profile form via `llvm-mipdata create` process once at build-time, which then we can merge with multiple Cs (raw counters) only via `llvm-mipdata merge`.

  DHeader:
  D1: 
    &CHeader - &D1(.)  // PC-relative to CHeader/Section
    &C1  - &D1(.) // PC-relative to C1
     // Subtract these two values to get the section-relative constant value -- &C1 - &CHeader =  (&C1  - &D1) - (&CHeader - &D1)
  D2:
    &CHeader - &D2(.) // PC-relative to CHeader/Section
    &C2 - &D2(.) // PC-relative to C2

This might be an optimized version by sharing the reference to CHeader in DHeader, which I meant in the prior comment.

  DHeader :
    &CHeader - &DHeader (.) // PC-relative to CHeader
  D1: 
    &C1 - &D1(.) // PC-relative to C1
     // We know this offset, &D1 - &DHeader from the layout in D. 
     // With the common value in DHeader, compute this base value &CHeader - &D1 = (&CHeader - &DHeader) - (&D1 - &DHeader)
     // Then derive the section-relative value in the same way --  &C1 - &CHeader =  (&C1  - &D1) - (&CHeader - &D1)
  D2:
    &C2 - &D2(.) // PC-relative to C2

I think a similar approach can be applied when multiple metadata sections exist. The key idea is to keep the distances among sections/regions (that will be in different address spaces) in the header, and use this common value + the layout offset to compute the section/region-relative constant value.
This last approach would largely maintain the size of metadata sections like D and thus allows them to be optionally extracted out. However, in the presence of multiple metadata sections, the complexity in dependencies (runtime and tools) seems high and also maintaining backward compatibility seems challenging.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D104060/new/

https://reviews.llvm.org/D104060