[all-commits] [llvm/llvm-project] b65267: [LV] Invalidate widening decisions after maximizin...

[David Green via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: b65267ca7bd160e4eb4853650c158058ec7a3ccb
      https://github.com/llvm/llvm-project/commit/b65267ca7bd160e4eb4853650c158058ec7a3ccb
  Author: David Green <david.green at arm.com>
  Date:   2022-03-31 (Thu, 31 Mar 2022)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    A llvm/test/Transforms/LoopVectorize/AArch64/maximize-bandwidth-invalidate.ll

  Log Message:
  -----------
  [LV] Invalidate widening decisions after maximizing vector bandwidth

When MaximizeVectorBandwidth is enabled, we can end up (via calls to
collectUniformsAndScalars/setCostBasedWideningDecision through
calculateRegisterUsage) making widening decisions before we have decided
whether to fold the tail by masking. These decisions will be wrong if we
later decided to fold the tail, for example when the trip count is very
low. It will use incorrect costs for loads that should get masked, using
standard memory operation costs instead.

This still at the moment uses the EmulatedMaskMemRefHack costs (a bit
unfortunately), but the old costs without this change were 1, leading to
too optimistic vectorization.

This slightly changes the way that the MaximizeVectorBandwidth option
works to make it easier to test, always honouring the option if it is
set.

Differential Revision: https://reviews.llvm.org/D120215