[llvm] [CodeGen] [EarlyIfConversion] Prioritize conversion of hard to predict branches (PR #174457)

[Jonathan Cohen via llvm-commits]

jcohen-apple wrote:

> What makes "values which are loaded from memory" hard to predict in general?

Let's look at some made up examples:

```
void SomeDSPFunction(float *signal, unsigned size) {

    for (i = 0; i < size; ++i) {
          float sample = signal[i];
          if (sample < threshold) {
              sample = someComputation(sample)
          }
          
          ....
    }
}
```

Compared to something like

```
void SomeBranchyFunc(enum param) {

    switch (param):

    ....
}
```

It's common in modern processors to perform branch prediction based on previous branch history (for more details look at how TAGE predictors work). The more consistent a branch's behavior is (both with itself and with nearby branches) the more likely  it is that the branch predictor will be able to predict the branch direction reliably. The predictor is correlating the current branch direction with previous control flow, but not with data.

In the first example the branch direction depends on some value we read from `signal` - using history would not be as useful in this case because we are branching based on _data_ that we have just recently read from memory, so it has not yet had a chance to affect control elsewhere (that we know of in the scope of that function). In the second case we use a function parameter to navigate through `SomeBranchyFunc`s control flow, and typically there will be some behavior in the caller or the larger branch history which correlates with the switch direction. Global history based branch predictors can really do well in this kind of scenario.

The key point is that the more program time that passes between when a value has been loaded from memory to when we reach the branch, the more chance it has to affect control flow elsewhere which can then be correlated with the direction of the branch we are currently trying to predict.

Of course there is a heuristic, so there are many counter examples we can come of with - but this aligns with what we have seen experimentally.  

https://github.com/llvm/llvm-project/pull/174457