[llvm] [VPlan] Dispatch to multiple exit blocks via middle blocks. (PR #112138)

[Florian Hahn via llvm-commits]

fhahn wrote:

> Hi @fhahn, whilst doing some more testing of my patches based on this version I came across what looks like a serious issue in `collectUsersInExitBlocks`. One of the programs in the LLVM test suite (yacr2) miscompiles and I tracked the problem down to some invalid ordering assumptions in `VPIRInstruction::generate`. Essentially `collectUsersInExitBlocks` adds operands to the VPIRInstruction in the order of BasicBlock predecessors in the scalar loop, whereas `VPIRInstruction::generate` follows the order of VPBasicBlock predecessors. The assumption here, which is incorrect, is that the exiting VPBasicBlock in the predecessor list at index `I` corresponds like-for-like with the exiting BasicBlock predecessor index `I` in the original scalar loop. For yacr2 this isn't true and the incoming PHI values got swapped the wrong way around.

Thanks for checking that's a good find! I added a test with a swapped use order for the exit block to check for this case and also improved the printing to include which VPBB the value is coming from.

Given that the current logic is tied to the existing legality checks which only support a single early exit, I updated the code in `collectUsersInExitBlocks` to iterate over the predecessors in VPlan and distinguish between the latch and early exit by checking if the VPlan predecessor is the middle.bb. WDYT?

Also adde `-enable-early-exit-vectorization` to an existing test to cover end-to-end IR codegen and slightly updated the legality test to check for the new message. Not sure if this patch should add the flag to all related tests? I removed the earlier tests from this PR that checked codegen, as we already have existing tests for that.

 

https://github.com/llvm/llvm-project/pull/112138