[all-commits] [llvm/llvm-project] 9c5504: [LV] Epilogue Vectorization with Optimal Control Flow

[Bardia Mahjour via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: 9c5504adceb544d9954ddb8ff3035a414f4b1423
      https://github.com/llvm/llvm-project/commit/9c5504adceb544d9954ddb8ff3035a414f4b1423
  Author: Bardia Mahjour <bmahjour at ca.ibm.com>
  Date:   2020-12-01 (Tue, 01 Dec 2020)

  Changed paths:
    M llvm/docs/Vectorizers.rst
    A llvm/docs/epilogue-vectorization-cfg.png
    M llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.h
    M llvm/lib/Transforms/Vectorize/LoopVectorize.cpp
    A llvm/test/Transforms/LoopVectorize/PowerPC/optimal-epilog-vectorization-profitability.ll
    A llvm/test/Transforms/LoopVectorize/PowerPC/optimal-epilog-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/X86/invariant-load-gather.ll
    M llvm/test/Transforms/LoopVectorize/X86/invariant-store-vectorization.ll
    M llvm/test/Transforms/LoopVectorize/X86/masked_load_store.ll
    A llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-limitations.ll
    A llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization-liveout.ll
    A llvm/test/Transforms/LoopVectorize/optimal-epilog-vectorization.ll

  Log Message:
  -----------
  [LV] Epilogue Vectorization with Optimal Control Flow

This is yet another attempt at providing support for epilogue
vectorization following discussions raised in RFC http://llvm.1065342.n5.nabble.com/llvm-dev-Proposal-RFC-Epilog-loop-vectorization-tt106322.html#none
and reviews D30247 and D88819.

Similar to D88819, this patch achieve epilogue vectorization by
executing a single vplan twice: once on the main loop and a second
time on the epilogue loop (using a different VF). However it's able
to handle more loops, and generates more optimal control flow for
cases where the trip count is too small to execute any code in vector
form.

Reviewed By: SjoerdMeijer

Differential Revision: https://reviews.llvm.org/D89566