[PATCH] D30062: Estimate speedup due to inlining and use that to adjust threshold.

[Easwaran Raman via Phabricator via llvm-commits]

eraman added a comment.

Sorry for the delay in collecting performance numbers. I now have some data to share.  First, some details on the methodology. I used ~400 microbenchmarks used internally at Google. I built them with the following percentage values of min-speedup-for-bonus: 0%, 5%, 10%, and 15%. I ran each benchmark 10 times in each configuration. Speedup/slowdown for a benchmark is calculated only when the p-value <=0.05 (and thus the results might include different subset of benchmarks for different configs). The numbers presented below are the geomean across all benchmarks.

Config | #Benchmarks | Geomean | #Slowdowns | #Speedups | Size increase percentage
0%   |   134 | 2.92% |  51 | 83 | 2.45%  
5%   |   121 | 1.05% |  41 | 80 | 1.58%
10% |   115 | 0.8%   |  51 | 64 | 1.32%
15% |   160 | 1.03% | 44 | 116| 1.02%

Some observations:

- The best geomean performance comes when the min-speedup-for-bonus is set to 0%. I interpret this to mean that it is generally a performance win to increase the threshold for hot callsites, and the speedup estimation is a way to control the size growth.
- The performance when the min-speedup-for-bonus is set to 10% sits in-between that of 5% and 15%.  As I mentioned above, these are not apples-to-apples comparisons beacause we compute geomean on a different set of benchmarks. Even for the same benchmark, it is possible (and it does happen) that the performance numbers are not monotonically decreasing as the min-speedup-for-bonus is increased.
- For comparison, I calculated the size growth if we simply apply a 3X multiplier to the threshold irrespective of the callsite frequency. The size increase is 9.7%.

I'm collecting SPEC numbers now. I've also fixed a bug in the code and will update the patch shortly.


https://reviews.llvm.org/D30062