[llvm] [llvm] Lower latency bonus threshold in function specialization. (PR #143954)

[Alexandros Lamprineas via llvm-commits]

labrinea wrote:

I agree, while CodeSize savings is a reliable metric, Latency is inaccurate. We use it in an attempt to model hotness of code, i.e loop nests etc. The compilation time will increase if more specializations trigger. The benchmark used to monitor such regressions is the compile time tracker (repository: https://github.com/nikic/llvm-compile-time-tracker, website: http://llvm-compile-time-tracker.com), which compiles the llvm test suite. The interesting configurations for this change would be non-LTO -O3 and LTO -O3. I would measure number of specializations before/after, and instruction count increase while compiling.

> I tried computing the total latency as a sum of latencies
of the instructions that belong to non-dead code (including
the instructions that would be executed had they not been
"eliminated" due to the constant propagation). This total
latency should identify the total cost of executing the function
with the given argument being dynamically equal to the tried
constant value. Then the latency bonus would be computed
as the ratio between the latency of the "eliminated" instructions
and the total latency.

> While looking at this, I did not really get why we compute the latency
bonus as a ratio of the latency of the "eliminated" instructions
and the code-size of the whole function. It did not make much sense
to me.

The latency bonus is just an approximation, neither of the above gives a good answer, actually they both give similarly inaccurate answers, but the second (currently implemented) is cheaper to compute.

Another idea would be to see check if there are any unhandled instructions we could teach the instruction cost visitor to handle. Last time I checked there wasn't anything expect for memory stores which are very difficult to track.

https://github.com/llvm/llvm-project/pull/143954