[llvm] [LoopUnroll] Rotate loop before unrolling inside of UnrollRuntimeLoopRemainder (PR #148243)

[via llvm-commits]

annamthomas wrote:


Some comments inline.

Thinking more about this, how about a slightly different approach (now that we can correct perform rotation in the `UnrollRuntimeLoopRemainder`):
- pass the fact that we need rotation from the callee (`UnrollRuntimeLoopRemainder`) to the caller. 
- Depending on the caller, let the caller decide if it wants to do rotation and then call `UnrollRuntimeLoopRemainder` once more after doing the rotation. 

This has 2 benefits:
- Reduces the chances of "rotation without unrolling" even further. We passed all the legality and performance considerations for runtime unrolling, except for this rotation need. You can do what @fhahn previously suggested: assume we rotated and do the remaining checks on the header's exit count, i.e. as if the loop was rotated.
- The caller can decide what to do with this (for example, we don't want unroll-and-jam to do anything with this). Similarly, if there are downstream users of unrolling, they wont get affected by the "unexpected" rotation. 

https://github.com/llvm/llvm-project/pull/148243