[PATCH] D54815: [ThinLTO] Add summary entries for index-based WPD

[Teresa Johnson via Phabricator via llvm-commits]

tejohnson added a comment.

Two clarifications below:

In D54815#1372125 <https://reviews.llvm.org/D54815#1372125>, @tejohnson wrote:

> Ping on review plus some numbers and a small improvement that I will upload shortly.
>
> I did some measurements to see how much overhead we add to the bitcode files for an important internal app. The overhead from adding the new records is pretty minor. Most of the increase over current non-WPD ThinLTO bitcode sizes comes from the addition of the type test instructions and the type test metadata - this just comes from enabling the options to get those in the IR (independent of which type of WPD we do we'll need those). Comparing the current hybrid WPD bitcode with the new summarized WPD bitcode, the bitcode files are actually smaller with the index summarized type info compared to that with the current hybrid WPD. I believe this is due to the overhead of splitting the LTO unit into two modules. And the size of the minimized index-only bitcode we create just for the distributed thin link process in our build system is significantly smaller with the new approach - this goes back to my finding that we aren't able to split the LTO unit in many cases so we end up having to put all the IR for some of these files into the regular LTO partition.
>
> All sizes below are in bytes. "minimized bitcode" means the additional index-only bitcode file for a distributed thin link emitted by clang with -fthin-link-bitcode (i.e. just contains symtab, summaries, strtab).


The minimized bitcode also contains anything that needs to be compiled in the regular LTO partition. Which for the hybrid WPD means all the vtables, plus all the IR in the case when we cannot split it but have vtable defs (which is why there is such a huge increase in that for the hybrid WPD below).

> ThinLTO baseline (no LTO unit, no WPD): 
>  aggregate bitcode size: 21464737248
>  aggregate minimized bitcode: 3251583939
> 
> ThinLTO hybrid WPD (current WPD implementation):
>  aggregate bitcode size: 22013358720 (2.6% bigger than baseline)
>  aggregate minimized bitcode: 7521978208 (1039% bigger than baseline)
> 
> ThinLTO index-only WPD (this patch):
>  aggregate bitcode size: 7521978208 (2.3% bigger than baseline, smaller than hybrid WPD)
>  aggregate minimized bitcode: 850885040 (28.6% bigger than baseline, much much smaller than current hybrid WPD)
> 
> Breaking down the increase in the "minimized bitcode" from this patch:
> 
> - 3% of increase is due to the additional VTable definition record fields (the vptr ref/offset pairs)
> - 13% is due to the new Type ID metadata records (was about 14% before my latest tweak to make those more efficient that I will upload shortly)
> - The remainder of the increase as I noted earlier is simply due to the additional IR from the type info (which is going to be there regardless of WPD scheme).

For the minimized bitcode the increased IR shows up because the symtab and strtab are much larger.

> So I don't think it is necessary to gate this on removing any redundancies of the vtable summaries with the IR bitcode, the increase from that is pretty small. And




Repository:
  rL LLVM

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D54815/new/

https://reviews.llvm.org/D54815