[all-commits] [llvm/llvm-project] 7344f3: [LoopVectorize] Add strict in-order reduction supp...

[kmclaughlin-arm via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 7344f3d39a0dc934b22ee43b4f92160fc3ea2db6
      https://github.com/llvm/llvm-project/commit/7344f3d39a0dc934b22ee43b4f92160fc3ea2db6
  Author: Kerry McLaughlin <kerry.mclaughlin at arm.com>
  Date:   2021-04-06 (Tue, 06 Apr 2021)

  Changed paths:
    M llvm/include/llvm/Analysis/IVDescriptors.h
    M llvm/include/llvm/Transforms/Utils/LoopUtils.h
    M llvm/lib/Analysis/IVDescriptors.cpp
    M llvm/lib/Transforms/Utils/LoopUtils.cpp
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    A llvm/test/Transforms/LoopVectorize/AArch64/strict-fadd.ll

  Log Message:
  -----------
  [LoopVectorize] Add strict in-order reduction support for fixed-width vectorization

Previously we could only vectorize FP reductions if fast math was enabled, as this allows us to
reorder FP operations. However, it may still be beneficial to vectorize the loop by moving
the reduction inside the vectorized loop and making sure that the scalar reduction value
be an input to the horizontal reduction, e.g:

  %phi = phi float [ 0.0, %entry ], [ %reduction, %vector_body ]
  %load = load <8 x float>
  %reduction = call float @llvm.vector.reduce.fadd.v8f32(float %phi, <8 x float> %load)

This patch adds a new flag (IsOrdered) to RecurrenceDescriptor and makes use of the changes added
by D75069 as much as possible, which already teaches the vectorizer about in-loop reductions.
For now in-order reduction support is off by default and controlled with the `-enable-strict-reductions` flag.

Reviewed By: david-arm

Differential Revision: https://reviews.llvm.org/D98435