[all-commits] [llvm/llvm-project] 87936c: [LoopVectorize] Fix assertion failure in fixReduct...

[david-arm via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 87936c7b131ee141a1309d5535f149ac48ff694e
      https://github.com/llvm/llvm-project/commit/87936c7b131ee141a1309d5535f149ac48ff694e
  Author: David Sherwood <david.sherwood at arm.com>
  Date:   2022-05-25 (Wed, 25 May 2022)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    A llvm/test/Transforms/LoopVectorize/AArch64/scalable-reductions-tf.ll
    M llvm/test/Transforms/LoopVectorize/AArch64/scalable-reductions.ll

  Log Message:
  -----------
  [LoopVectorize] Fix assertion failure in fixReduction when tail-folding

When compiling the attached new test in scalable-reductions-tf.ll we
were hitting this assertion in fixReduction:

  Assertion `isa<PHINode>(U) && "Reduction exit must feed Phi's or select"

The loop contains a reduction and an intermediate store of the reduction
value. When vectorising with tail-folding the contains of 'U' in the
assertion above happened to be a scatter_store. It turns out that we
were still creating a widen recipe for the invariant store, despite
knowing that we can actually sink it. The simplest fix is to change
buildVPlanWithVPRecipes so that we look for invariant stores before
attempting to widen it.

Differential Revision: https://reviews.llvm.org/D126295