[llvm] [MemDep] Optimize SortNonLocalDepInfoCache sorting strategy for large caches with few unsorted entries (PR #143107)

[via llvm-commits]

DingdWang wrote:

Compile time result: https://llvm-compile-time-tracker.com/compare.php?from=26f3f24a4f0a67eb23d255aba7a73a12bee1db11&to=4e570371a96e25c8b7f9d25266afe864dcfdfa20&stat=instructions%3Au

The optimization effect does not appear significant on the benchmark. I checked the related cache size and NumSortedEntries outputs, and found that on the benchmark, the overall cache size is relatively small—around a few hundred entries—and the function is called infrequently. As a result, the number of times this patch can actually hit and optimize is limited, so the performance improvement is not obvious.
In contrast, for large files, this function can be called nearly 16 million times, with about 1.5 million hits benefiting from the optimization, which leads to a clear performance gain. This data suggests that the original code’s approach of only considering cases 1 and 2 for insertion sort is insufficient; in reality, cases 3, 4, and 5 occur frequently as well.

statistics of sqlit3 from benchmark
```
Total: 95321

Cache.size statistics:
  Max: 613
  Min: 0
  Median: 3
  Average: 29.8503

Cache.size - NumSortedEntries difference distribution (top 20 by frequency), excluding NumSortedEntries=0 and diff=0:
  1: 6133
  2: 1650
  3: 1147
  5: 1088
  4: 626
  7: 620
  6: 490
  8: 388
  10: 372
  9: 231
  11: 204
  12: 140
  13: 100
  19: 88
  15: 84
  14: 82
  16: 72
  18: 69
  17: 67
  22: 65

Number of cases where 2 < Cache.size - NumSortedEntries < log2(Cache.size): 1784

Difference distribution in special cases (top 20), excluding NumSortedEntries=0 and diff=0:
  3: 703
  5: 500
  4: 300
  6: 146
  7: 115
  8: 20

Cache.size statistics for top 10 difference values in special cases:
  Difference = 3:
    Count: 703
    Max Cache.size: 571
    Min Cache.size: 9
    Median Cache.size: 34
    Average Cache.size: 86.0484
  Difference = 5:
    Count: 500
    Max Cache.size: 589
    Min Cache.size: 33
    Median Cache.size: 96.5
    Average Cache.size: 156.2580
  Difference = 4:
    Count: 300
    Max Cache.size: 600
    Min Cache.size: 17
    Median Cache.size: 64.0
    Average Cache.size: 109.3567
  Difference = 6:
    Count: 146
    Max Cache.size: 606
    Min Cache.size: 66
    Median Cache.size: 197.5
    Average Cache.size: 220.3151
  Difference = 7:
    Count: 115
    Max Cache.size: 613
    Min Cache.size: 130
    Median Cache.size: 250
    Average Cache.size: 293.1391
  Difference = 8:
    Count: 20
    Max Cache.size: 440
    Min Cache.size: 257
    Median Cache.size: 326.0
    Average Cache.size: 347.1500
```

statistics of a big file
```
Total: 15971872

Cache.size statistics:
  Max: 57548
  Min: 0
  Median: 165.0
  Average: 3376.9060

Cache.size - NumSortedEntries difference distribution (top 20 by frequency), excluding NumSortedEntries=0 and diff=0:
  2: 2339040
  1: 700840
  4: 502824
  6: 302040
  5: 289494
  7: 168130
  3: 159230
  8: 46549
  9: 24162
  10: 22767
  11: 13970
  12: 4347
  13: 2280
  161: 1112
  160: 1098
  157: 1092
  162: 1083
  158: 1015
  155: 989
  154: 954

Number of cases where 2 < Cache.size - NumSortedEntries < log2(Cache.size): 1526693

Difference distribution in special cases (top 20), excluding NumSortedEntries=0 and diff=0:
  4: 502647
  6: 301711
  5: 288484
  7: 167896
  3: 158712
  8: 45461
  9: 23584
  10: 21185
  11: 12859
  12: 2953
  13: 1074
  14: 127

Cache.size statistics for top 10 difference values in special cases:
  Difference = 4:
    Count: 502647
    Max Cache.size: 57432
    Min Cache.size: 17
    Median Cache.size: 11537
    Average Cache.size: 11473.2582
  Difference = 6:
    Count: 301711
    Max Cache.size: 56986
    Min Cache.size: 65
    Median Cache.size: 11535
    Average Cache.size: 11426.9020
  Difference = 5:
    Count: 288484
    Max Cache.size: 57400
    Min Cache.size: 50
    Median Cache.size: 10694.0
    Average Cache.size: 10912.9279
  Difference = 7:
    Count: 167896
    Max Cache.size: 57382
    Min Cache.size: 129
    Median Cache.size: 10765.5
    Average Cache.size: 10981.8878
  Difference = 3:
    Count: 158712
    Max Cache.size: 57219
    Min Cache.size: 9
    Median Cache.size: 11095.0
    Average Cache.size: 11065.3424
  Difference = 8:
    Count: 45461
    Max Cache.size: 57326
    Min Cache.size: 257
    Median Cache.size: 11178
    Average Cache.size: 11267.4616
  Difference = 9:
    Count: 23584
    Max Cache.size: 57272
    Min Cache.size: 513
    Median Cache.size: 11668.0
    Average Cache.size: 11672.4953
  Difference = 10:
    Count: 21185
    Max Cache.size: 57442
    Min Cache.size: 1025
    Median Cache.size: 13721
    Average Cache.size: 14924.2003
  Difference = 11:
    Count: 12859
    Max Cache.size: 48734
    Min Cache.size: 2051
    Median Cache.size: 13343
    Average Cache.size: 13009.9933
  Difference = 12:
    Count: 2953
    Max Cache.size: 39377
    Min Cache.size: 4118
    Median Cache.size: 14387
    Average Cache.size: 14023.0687
```

https://github.com/llvm/llvm-project/pull/143107