[llvm] [LV] Add -fold-epilogue-tail option for tail-folded epilogue (PR #190697)
Hassnaa Hamdi via llvm-commits
llvm-commits at lists.llvm.org
Sun May 3 06:52:16 PDT 2026
================
@@ -198,12 +198,17 @@ static cl::opt<unsigned> VectorizeMemoryCheckThreshold(
"vectorize-memory-check-threshold", cl::init(128), cl::Hidden,
cl::desc("The maximum allowed number of runtime memory checks"));
-/// Option tail-folding-policy indicates that an epilogue is undesired, that
-/// tail folding is preferred, and this lists all options. I.e., the vectorizer
-/// will try to fold the tail-loop (epilogue) into the vector body and predicate
-/// the instructions accordingly. If tail-folding fails, there are different
-/// fallback strategies depending on these values:
-enum class TailFoldingPolicyTy { None = 0, PreferFoldTail, MustFoldTail };
+/// Option tail-folding-policy controls the tail-folding strategy and lists all
+/// available options. The vectorizer will attempt to fold the tail-loop into
+/// the vector loop (main/epilogue loops) and predicate the instructions
+/// accordingly. If tail-folding fails, there are different fallback strategies
+/// depending on these values:
+enum class TailFoldingPolicyTy {
----------------
hassnaaHamdi wrote:
Hi,
Thanks for the detailed comment.
Here are my opinions:
Regarding to this:
> Should a MustFoldTail option also apply to the epilog loop, i.e., MustFoldEpilogTail, in addition to its PreferFoldEpilogTail option, or does "must" aim to retain a single (main) loop, to limit code size.
I think we shouldn't care a lot about applying different options on the epilogue because the code eventually/in-the-near-future should depend on the costs to dynamically choose best vectorization style.
Regarding to this:
> Does None mean "must-not-fold-tail" as in "no-folding", as opposed to "no-policy-specified"? Will None now mean both main and epilog must not fold their tail, while PreferFoldEpilogTail implies main loop (only) must not fold its tail.
There are other factors/hints that could restrict the policy flag. For main loop, according to the current logic, we could still end up folding the tail even if the policy prevents folding. Also because of those other hints, having `None` option could be different from no-policy-specified and could be same because we still do checks on other hints.
For epilogue loop, `PreferFoldEpilogTail` doesn't imply main loop must not fold its tail, but it implies that we prefer folding the epilogue tail (whenever it exists), so that option only affects the epilogue not the main loop.
https://github.com/llvm/llvm-project/pull/190697
More information about the llvm-commits
mailing list