[PATCH] D120230: [SelectOpti][1/4] Setup new select-optimize pass

Tue Mar 15 19:14:49 PDT 2022

apostolakis added a comment.

In D120230#3381419 <https://reviews.llvm.org/D120230#3381419>, @wenlei wrote:

>>> So, if you are interested in Sample-PGO it is better to wait for the next patch series that will tailor this pass for that. At that point, we can iterate and refine to make sure that the pass addresses these suboptimal cases and avoids regressions for your workloads.
>>
>> The lack of mispredict data for cmovs will be a problem but we will not be stuck with a cmov decision but rather we might observe some oscillations (which is still problematic but not atypical of SampleFDO settings and there are some known remedies).
>
> Say we end up with cmov in one of the sample PGO iterations (either due to lack of profile, or profile indicating branch being unbiased), we would lose the control flow profile that is needed to tell how biased that original branch is, because we've turned that control flow into data flow. Unless we never use cmov for branches without profile info, we could keep generating cmov in future iterations even if branch becomes more biased later because we will never get control flow profile again.

We will not get stuck if the non-profile logic and default values for branch weights and mispredict rates are set in a way that favors branches. Already talked about mispredict rates. Another example is the base heuristic that converts selects to branches when there is an expensive operand in the computation of the cold operand (currently less than 20% selected operand). If this select is currently a branch and the profile-based weight is 25% for the expensive operand (so not cold) then we might convert that to a cmov. Then we will not have profile information but we can allow all selects with expensive operands but no profile data to be conservatively converted to branches, and then if the expensive operand becomes cold the select will become a branch, otherwise there will be oscillation.

In general, I think some reasonable non-profile-based heuristics that favor branches will be better than blindly converting all cases to branches even for cases where a cmov seems preferable even with unfavorable branch weights and mispredict rate.

In D120230#3381419 <https://reviews.llvm.org/D120230#3381419>, @wenlei wrote:

> In D120230#3380072 <https://reviews.llvm.org/D120230#3380072>, @apostolakis wrote:
>
>> The default misprediction rate used by the compiler (currently 25%) is expected to be less than the threshold that motivates a conversion to a cmov based on mispredict data. So, for example if a branch mispredicts 50% of the time, we could convert that to a cmov. Then the cmov will get compared with a branch that mispredicts 25% of the time, making the branch perhaps more desirable than it would have been if we had mispredict data. It is not necessary that the rest of the heuristics will allow a conversion back to a branch, but the cmov decision will be for sure revertible.
>
> nit: saying misprediction rate here and in the RFC is a bit confusing because today we don't have that data in profile. that threshold is how biased a branch is, which is a proxy for branch miss. But branch predictor could still do well (low branch miss) for unbiased branches.

Just to clarify since you misunderstood me, I was referring (both here and in the RFC) to actual misprediction rate and not branch weights. If you look at the 3rd patch (D120232 <https://reviews.llvm.org/D120232>) that includes the loop-level heuristics you will see that a mispredict rate (that conservatively defaults to 25%) is taken into account to calculate the cost of branches. Branch weights are used to compute the branch cost for correctly predicted cases (BranchCost = PredictedPathCost + MispredictCost * MispredictRate, where PredictedPathCost = TrueOpCost * TrueWeight + FalseOpCost * FalseWeight). Unbiased branches indeed do not necessarily lead to mispredictions (as discussed in the RFC) and the only case where branch weights are used for predictability for this optimization is for cases where the branch is entirely biased to one direction and thus it is somewhat expected to be predictable (part of the base heuristics). If profile-based mispredict rates were available, there would at least one extra base heuristic that handles some extreme cases (highly or poorly predicted) and then for in-between cases the existing heuristics will be refined to make the decision.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D120230/new/

https://reviews.llvm.org/D120230