[PATCH] D54815: [ThinLTO] Add summary entries for index-based WPD

[Teresa Johnson via Phabricator via llvm-commits]

tejohnson added a comment.

Ping on review plus some numbers and a small improvement that I will upload shortly.

I did some measurements to see how much overhead we add to the bitcode files for an important internal app. The overhead from adding the new records is pretty minor. Most of the increase over current non-WPD ThinLTO bitcode sizes comes from the addition of the type test instructions and the type test metadata - this just comes from enabling the options to get those in the IR (independent of which type of WPD we do we'll need those). Comparing the current hybrid WPD bitcode with the new summarized WPD bitcode, the bitcode files are actually smaller with the index summarized type info compared to that with the current hybrid WPD. I believe this is due to the overhead of splitting the LTO unit into two modules. And the size of the minimized index-only bitcode we create just for the distributed thin link process in our build system is significantly smaller with the new approach - this goes back to my finding that we aren't able to split the LTO unit in many cases so we end up having to put all the IR for some of these files into the regular LTO partition.

All sizes below are in bytes. "minimized bitcode" means the additional index-only bitcode file for a distributed thin link emitted by clang with -fthin-link-bitcode (i.e. just contains symtab, summaries, strtab).

ThinLTO baseline (no LTO unit, no WPD): 
aggregate bitcode size: 21464737248
aggregate minimized bitcode: 3251583939

ThinLTO hybrid WPD (current WPD implementation):
aggregate bitcode size: 22013358720 (2.6% bigger than baseline)
aggregate minimized bitcode: 7521978208 (1039% bigger than baseline)

ThinLTO index-only WPD (this patch):
aggregate bitcode size: 7521978208 (2.3% bigger than baseline, smaller than hybrid WPD)
aggregate minimized bitcode: 850885040 (28.6% bigger than baseline, much much smaller than current hybrid WPD)

Breaking down the increase in the "minimized bitcode" from this patch:

- 3% of increase is due to the additional VTable definition record fields (the vptr ref/offset pairs)
- 13% is due to the new Type ID metadata records (was about 14% before my latest tweak to make those more efficient that I will upload shortly)
- The remainder of the increase as I noted earlier is simply due to the additional IR from the type info (which is going to be there regardless of WPD scheme).

So I don't think it is necessary to gate this on removing any redundancies of the vtable summaries with the IR bitcode, the increase from that is pretty small. And


Repository:
  rL LLVM

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D54815/new/

https://reviews.llvm.org/D54815