[llvm] f86d707 - [LoopVectorize] Reland: Add metadata to distinguish vectorized loop body from scalar remainder (#194912)
via llvm-commits
llvm-commits at lists.llvm.org
Fri May 1 07:01:05 PDT 2026
Author: Adel Ejjeh
Date: 2026-05-01T09:00:59-05:00
New Revision: f86d707db6a1c91f65ef32e5afff9d566948b4ad
URL: https://github.com/llvm/llvm-project/commit/f86d707db6a1c91f65ef32e5afff9d566948b4ad
DIFF: https://github.com/llvm/llvm-project/commit/f86d707db6a1c91f65ef32e5afff9d566948b4ad.diff
LOG: [LoopVectorize] Reland: Add metadata to distinguish vectorized loop body from scalar remainder (#194912)
Reland of #190258, reverted in #194901 due to a null-pointer dereference
when the scalar preheader is absent. Fixed by guarding the metadata
emission with a null check on `ScalarPH`. Also adds a dedicated test
(`vectorize-loop-kind-metadata.ll`) verifying the metadata is emitted
only when remarks are enabled.
---
Add two new loop metadata attributes — `llvm.loop.vectorize.body` and
`llvm.loop.vectorize.epilogue` — that the loop vectorizer sets on the
generated vector loop and epilogue loop respectively. The metadata is
only emitted when optimization remarks are enabled (`ORE->enabled()`),
so it has zero cost in normal compilation.
These enable downstream passes (LoopUnroll, WarnMissedTransforms) to
produce more precise optimization remarks. Instead of the generic "loop
not unrolled" warning on a source line that was vectorized, the unroller
can now report:
- **"vectorized loop"** for the main vector body
- **"epilogue loop"** for the scalar epilogue/remainder
- **"epilogue vectorized loop"** for an epilogue that was itself
vectorized during epilogue vectorization (carries both attributes)
A shared `getLoopVectorizeKindPrefix()` helper in
`LoopUtils.h`/`LoopUtils.cpp` reads the metadata and returns the
appropriate prefix string, used by both `LoopUnroll.cpp` and
`WarnMissedTransforms.cpp`. The metadata emission in `VPlan.cpp` uses
`Loop::addIntLoopAttribute` from the NFC PR #194676.
Two end-to-end tests exercise the full `loop-vectorize → loop-unroll`
pipeline with forced epilogue vectorization
(`-enable-epilogue-vectorization -epilogue-vectorization-force-VF=4`) to
produce all four loop categories from a single vectorizable function.
Each test also includes a plain (non-vectorized) function to cover the
baseline "loop" case. Both tests verify stderr diagnostic output and
YAML structured remarks.
**`LoopUnroll/vectorizer-loop-kind-remarks.ll`** checks for
successful-unroll remarks.
**`LoopTransformWarning/vectorizer-loop-kind-unroll-warning.ll`** checks
for failed-unroll warnings.
AI Disclaimer: this patch was generated with assistance of GitHub
Copilot/Claude Opus and reviewed by a human.
Added:
llvm/test/Transforms/LoopTransformWarning/vectorizer-loop-kind-unroll-warning.ll
llvm/test/Transforms/LoopUnroll/vectorizer-loop-kind-remarks.ll
llvm/test/Transforms/LoopVectorize/vectorize-loop-kind-metadata.ll
Modified:
llvm/docs/LangRef.rst
llvm/docs/TransformMetadata.rst
llvm/include/llvm/Transforms/Utils/LoopUtils.h
llvm/lib/Transforms/Scalar/WarnMissedTransforms.cpp
llvm/lib/Transforms/Utils/LoopUnroll.cpp
llvm/lib/Transforms/Utils/LoopUtils.cpp
llvm/lib/Transforms/Vectorize/VPlan.cpp
llvm/test/Transforms/LoopVectorize/RISCV/remark-reductions.ll
llvm/test/Transforms/LoopVectorize/remarks-reduction-inloop.ll
Removed:
################################################################################
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 83206c76a0954..b9dc46b80f873 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -7976,6 +7976,48 @@ Attributes in the metadata will be added to both the vectorized and
epilogue loop.
See :ref:`Transformation Metadata <transformation-metadata>` for details.
+'``llvm.loop.vectorize.body``' Metadata
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This metadata is automatically added by the loop vectorizer to the
+vectorized loop body. It is used by subsequent optimization passes
+(such as the loop unroller and ``WarnMissedTransforms``) to emit more
+precise optimization remarks that identify the loop as a vector loop.
+
+The first operand is the string
+``llvm.loop.vectorize.body`` and the second operand is an
+integer. A value of 1 indicates this is a vectorized loop body:
+
+.. code-block:: llvm
+
+ !0 = !{!"llvm.loop.vectorize.body", i32 1}
+
+'``llvm.loop.vectorize.epilogue``' Metadata
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This metadata is automatically added by the loop vectorizer to the
+scalar remainder (epilogue) loop to identify it as such. It is used by
+subsequent optimization passes (such as the loop unroller and
+``WarnMissedTransforms``) to emit more precise optimization remarks
+that distinguish between the vectorized loop and its scalar remainder.
+
+Together these two attributes provide a four-way classification:
+
+- ``body`` only: main vectorized loop body
+- ``epilogue`` only: scalar epilogue loop after vectorization
+- Both ``body`` and ``epilogue``: vectorized epilogue
+ (a remainder loop that was itself vectorized during epilogue
+ vectorization)
+- Neither: a plain loop not produced by the vectorizer
+
+The first operand is the string
+``llvm.loop.vectorize.epilogue`` and the second operand is
+an integer. A value of 1 indicates the loop is a remainder loop:
+
+.. code-block:: llvm
+
+ !0 = !{!"llvm.loop.vectorize.epilogue", i32 1}
+
'``llvm.loop.unroll``'
^^^^^^^^^^^^^^^^^^^^^^
diff --git a/llvm/docs/TransformMetadata.rst b/llvm/docs/TransformMetadata.rst
index 1f9f50f1f33bc..700761a383e9a 100644
--- a/llvm/docs/TransformMetadata.rst
+++ b/llvm/docs/TransformMetadata.rst
@@ -168,6 +168,27 @@ has unroll metadata).
The attributes specified by ``llvm.loop.vectorize.followup_all`` are
added to both loops.
+In addition to the above, the vectorizer automatically annotates the
+generated loops with two metadata attributes for remark quality:
+
+- ``llvm.loop.vectorize.body`` is added to the vectorized loop.
+- ``llvm.loop.vectorize.epilogue`` is added to the scalar
+ remainder (epilogue) loop.
+
+Together these provide a four-way classification:
+
+- ``body`` only: main vectorized loop body
+- ``epilogue`` only: scalar epilogue loop after vectorization
+- Both: vectorized epilogue (a remainder that was itself
+ vectorized during epilogue vectorization)
+- Neither: a plain loop not produced by the vectorizer
+
+This is used by subsequent passes (e.g., the loop unroller and
+``WarnMissedTransforms``) to produce more precise optimization remarks.
+For instance, instead of reporting both "loop unrolled" and "loop not
+unrolled" for the same source line, the unroller can clarify that a
+*vectorized* loop was unrolled while its *scalar epilogue* was not.
+
When using a follow-up attribute, it replaces any automatically deduced
attributes for the generated loop in question. Therefore it is
recommended to add ``llvm.loop.isvectorized`` to
diff --git a/llvm/include/llvm/Transforms/Utils/LoopUtils.h b/llvm/include/llvm/Transforms/Utils/LoopUtils.h
index f10f437eeb6a8..5775d106c8453 100644
--- a/llvm/include/llvm/Transforms/Utils/LoopUtils.h
+++ b/llvm/include/llvm/Transforms/Utils/LoopUtils.h
@@ -306,6 +306,13 @@ enum TransformationMode {
TM_SuppressedByUser = TM_Disable | TM_Force
};
+/// Return a short prefix describing the loop's vectorizer origin based on
+/// the \c llvm.loop.vectorize.body and \c llvm.loop.vectorize.epilogue
+/// metadata. The result is one of \c "vectorized epilogue ", \c "vectorized ",
+/// \c "epilogue ", or \c "" (empty) and is intended to be prepended to
+/// loop-kind tokens in optimization remarks.
+LLVM_ABI StringRef getLoopVectorizeKindPrefix(const Loop *L);
+
/// @{
/// Get the mode for LLVM's supported loop transformations.
LLVM_ABI TransformationMode hasUnrollTransformation(const Loop *L);
diff --git a/llvm/lib/Transforms/Scalar/WarnMissedTransforms.cpp b/llvm/lib/Transforms/Scalar/WarnMissedTransforms.cpp
index e53019768e881..ac19f5d17ab87 100644
--- a/llvm/lib/Transforms/Scalar/WarnMissedTransforms.cpp
+++ b/llvm/lib/Transforms/Scalar/WarnMissedTransforms.cpp
@@ -25,13 +25,20 @@ static void warnAboutLeftoverTransformations(Loop *L,
OptimizationRemarkEmitter *ORE) {
if (hasUnrollTransformation(L) == TM_ForcedByUser) {
LLVM_DEBUG(dbgs() << "Leftover unroll transformation\n");
+
+ // Determine whether this loop originated from the vectorizer so we can
+ // produce more informative remarks.
+ StringRef LoopKind = getLoopVectorizeKindPrefix(L);
+
ORE->emit(
DiagnosticInfoOptimizationFailure(DEBUG_TYPE,
"FailedRequestedUnrolling",
L->getStartLoc(), L->getHeader())
- << "loop not unrolled: the optimizer was unable to perform the "
- "requested transformation; the transformation might be disabled or "
- "specified as part of an unsupported transformation ordering");
+ << LoopKind.str() +
+ "loop not unrolled: the optimizer was unable to perform the "
+ "requested transformation; the transformation might be disabled "
+ "or "
+ "specified as part of an unsupported transformation ordering");
}
if (hasUnrollAndJamTransformation(L) == TM_ForcedByUser) {
diff --git a/llvm/lib/Transforms/Utils/LoopUnroll.cpp b/llvm/lib/Transforms/Utils/LoopUnroll.cpp
index 45b70fd148ead..e17b4e134eaca 100644
--- a/llvm/lib/Transforms/Utils/LoopUnroll.cpp
+++ b/llvm/lib/Transforms/Utils/LoopUnroll.cpp
@@ -825,6 +825,11 @@ llvm::UnrollLoop(Loop *L, UnrollLoopOptions ULO, LoopInfo *LI,
}
using namespace ore;
+
+ // Determine whether this loop originated from the vectorizer so we can
+ // produce more informative remarks.
+ StringRef LoopKind = getLoopVectorizeKindPrefix(L);
+
// Report the unrolling decision.
if (CompletelyUnroll) {
LLVM_DEBUG(dbgs() << "COMPLETELY UNROLLING loop %" << Header->getName()
@@ -833,7 +838,7 @@ llvm::UnrollLoop(Loop *L, UnrollLoopOptions ULO, LoopInfo *LI,
ORE->emit([&]() {
return OptimizationRemark(DEBUG_TYPE, "FullyUnrolled", L->getStartLoc(),
L->getHeader())
- << "completely unrolled loop with "
+ << "completely unrolled " + LoopKind.str() + "loop with "
<< NV("UnrollCount", ULO.Count) << " iterations";
});
} else {
@@ -851,7 +856,8 @@ llvm::UnrollLoop(Loop *L, UnrollLoopOptions ULO, LoopInfo *LI,
ORE->emit([&]() {
OptimizationRemark Diag(DEBUG_TYPE, "PartialUnrolled", L->getStartLoc(),
L->getHeader());
- Diag << "unrolled loop by a factor of " << NV("UnrollCount", ULO.Count);
+ Diag << "unrolled " + LoopKind.str() + "loop by a factor of "
+ << NV("UnrollCount", ULO.Count);
if (ULO.Runtime)
Diag << " with run-time trip count"
<< (ULO.UnrollRemainder ? " (remainder unrolled)" : "");
diff --git a/llvm/lib/Transforms/Utils/LoopUtils.cpp b/llvm/lib/Transforms/Utils/LoopUtils.cpp
index 9c7d10629adf6..03b693974e5fe 100644
--- a/llvm/lib/Transforms/Utils/LoopUtils.cpp
+++ b/llvm/lib/Transforms/Utils/LoopUtils.cpp
@@ -353,6 +353,18 @@ bool llvm::hasDisableLICMTransformsHint(const Loop *L) {
return getBooleanLoopAttribute(L, LLVMLoopDisableLICM);
}
+StringRef llvm::getLoopVectorizeKindPrefix(const Loop *L) {
+ bool IsVectorBody = getBooleanLoopAttribute(L, "llvm.loop.vectorize.body");
+ bool IsEpilogue = getBooleanLoopAttribute(L, "llvm.loop.vectorize.epilogue");
+ if (IsVectorBody && IsEpilogue)
+ return "vectorized epilogue ";
+ if (IsVectorBody)
+ return "vectorized ";
+ if (IsEpilogue)
+ return "epilogue ";
+ return "";
+}
+
TransformationMode llvm::hasUnrollTransformation(const Loop *L) {
if (getBooleanLoopAttribute(L, "llvm.loop.unroll.disable"))
return TM_SuppressedByUser;
diff --git a/llvm/lib/Transforms/Vectorize/VPlan.cpp b/llvm/lib/Transforms/Vectorize/VPlan.cpp
index f1d96443e1283..91fa4c57e5bf9 100644
--- a/llvm/lib/Transforms/Vectorize/VPlan.cpp
+++ b/llvm/lib/Transforms/Vectorize/VPlan.cpp
@@ -31,6 +31,7 @@
#include "llvm/ADT/Twine.h"
#include "llvm/Analysis/DomTreeUpdater.h"
#include "llvm/Analysis/LoopInfo.h"
+#include "llvm/Analysis/OptimizationRemarkEmitter.h"
#include "llvm/IR/BasicBlock.h"
#include "llvm/IR/CFG.h"
#include "llvm/IR/IRBuilder.h"
@@ -1793,6 +1794,11 @@ void LoopVectorizationPlanner::updateLoopMetadataAndProfileInfo(
Hints.setAlreadyVectorized();
}
}
+ // Tag the scalar remainder so downstream passes (e.g. the unroller and
+ // WarnMissedTransforms) can produce more informative remarks. Only emit
+ // when remarks are enabled.
+ if (ORE->enabled() && ScalarPH && ScalarPH->hasPredecessors())
+ OrigLoop->addIntLoopAttribute("llvm.loop.vectorize.epilogue", 1);
if (!VectorLoop)
return;
@@ -1813,6 +1819,10 @@ void LoopVectorizationPlanner::updateLoopMetadataAndProfileInfo(
Hints.setAlreadyVectorized();
}
}
+ // Tag the vector loop body so downstream passes can identify it. Only
+ // emit when remarks are enabled.
+ if (ORE->enabled())
+ VectorLoop->addIntLoopAttribute("llvm.loop.vectorize.body", 1);
TargetTransformInfo::UnrollingPreferences UP;
TTI.getUnrollingPreferences(VectorLoop, *PSE.getSE(), UP, ORE);
if (!UP.UnrollVectorizedLoop || VectorizingEpilogue)
diff --git a/llvm/test/Transforms/LoopTransformWarning/vectorizer-loop-kind-unroll-warning.ll b/llvm/test/Transforms/LoopTransformWarning/vectorizer-loop-kind-unroll-warning.ll
new file mode 100644
index 0000000000000..0168bddacca51
--- /dev/null
+++ b/llvm/test/Transforms/LoopTransformWarning/vectorizer-loop-kind-unroll-warning.ll
@@ -0,0 +1,120 @@
+; RUN: opt < %s -passes='loop-vectorize,loop-unroll,transform-warning' \
+; RUN: -force-vector-width=4 -force-vector-interleave=1 \
+; RUN: -enable-epilogue-vectorization -epilogue-vectorization-force-VF=4 \
+; RUN: -disable-output -pass-remarks-missed=transform-warning 2>&1 | FileCheck %s
+; RUN: opt < %s -passes='loop-vectorize,loop-unroll,transform-warning' \
+; RUN: -force-vector-width=4 -force-vector-interleave=1 \
+; RUN: -enable-epilogue-vectorization -epilogue-vectorization-force-VF=4 \
+; RUN: -disable-output -pass-remarks-output=%t.yaml 2>&1
+; RUN: FileCheck --input-file=%t.yaml %s --check-prefix=YAML
+
+; Verify that when the loop vectorizer produces body / epilogue
+; metadata and the unroller *fails* to unroll (runtime unrolling is disabled),
+; the surviving unroll hints reach WarnMissedTransforms, which emits a
+; distinct "not unrolled" warning for each loop-kind qualifier.
+;
+; Pipeline: loop-vectorize -> loop-unroll -> transform-warning.
+; Epilogue vectorization is forced to exercise all four loop categories:
+;
+; 1. plain loop – not touched by the vectorizer
+; 2. vectorized loop – main vector body (has vectorize.body)
+; 3. epilogue loop – epilogue loop (has vectorize.epilogue)
+;
+; Runtime unrolling is disabled via metadata so the unroller cannot act,
+; causing the pragma-unroll hint to survive to the warning pass.
+; Both stderr warnings and YAML structured output are checked.
+
+;--- plain_loop: vectorize.enable=false, runtime unroll disabled ---------------
+
+; CHECK: warning: test.cpp:1:1: loop not unrolled: the optimizer was unable to perform the requested transformation
+; CHECK-NOT: warning: test.cpp:1:1: vectorized
+; CHECK-NOT: warning: test.cpp:1:1: epilogue
+
+define void @plain_loop(ptr noalias %A, i64 %n) !dbg !100 {
+entry:
+ br label %loop, !dbg !108
+loop:
+ %i = phi i64 [0, %entry], [%i.next, %loop]
+ %gep = getelementptr inbounds i32, ptr %A, i64 %i
+ %v = load i32, ptr %gep, align 4
+ %add = add i32 %v, 1
+ store i32 %add, ptr %gep, align 4
+ %i.next = add nuw nsw i64 %i, 1
+ %cmp = icmp slt i64 %i.next, %n
+ br i1 %cmp, label %loop, label %exit, !dbg !108, !llvm.loop !10
+exit:
+ ret void
+}
+
+;--- vectorizable_loop: vectorized with epilogue, all three fail to unroll -----
+
+; CHECK-DAG: warning: test.cpp:10:1: vectorized loop not unrolled: the optimizer was unable to perform the requested transformation
+; CHECK-DAG: warning: test.cpp:10:1: epilogue loop not unrolled: the optimizer was unable to perform the requested transformation
+
+define void @vectorizable_loop(ptr noalias %A, ptr noalias %B, i64 %n) !dbg !200 {
+entry:
+ br label %loop, !dbg !208
+loop:
+ %i = phi i64 [0, %entry], [%i.next, %loop]
+ %gep.a = getelementptr inbounds i32, ptr %A, i64 %i
+ %gep.b = getelementptr inbounds i32, ptr %B, i64 %i
+ %v = load i32, ptr %gep.a, align 4
+ %add = add i32 %v, 1
+ store i32 %add, ptr %gep.b, align 4
+ %i.next = add nuw nsw i64 %i, 1
+ %cmp = icmp slt i64 %i.next, %n
+ br i1 %cmp, label %loop, label %exit, !dbg !208, !llvm.loop !20
+exit:
+ ret void
+}
+
+;--- YAML checks ---------------------------------------------------------------
+
+; YAML: --- !Failure
+; YAML: Pass: transform-warning
+; YAML: Name: FailedRequestedUnrolling
+; YAML: Function: plain_loop
+; YAML: Args:
+; YAML: - String: 'loop not unrolled:
+
+; YAML: --- !Failure
+; YAML: Pass: transform-warning
+; YAML: Name: FailedRequestedUnrolling
+; YAML: Function: vectorizable_loop
+; YAML: Args:
+; YAML: - String: 'vectorized loop not unrolled:
+
+; YAML: --- !Failure
+; YAML: Pass: transform-warning
+; YAML: Name: FailedRequestedUnrolling
+; YAML: Function: vectorizable_loop
+; YAML: Args:
+; YAML: - String: 'epilogue loop not unrolled:
+
+;--- Metadata ------------------------------------------------------------------
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!2, !3}
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: NoDebug)
+!1 = !DIFile(filename: "test.cpp", directory: ".")
+!2 = !{i32 2, !"Dwarf Version", i32 4}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!99 = !DISubroutineType(types: !{})
+
+; Plain loop: vectorize disabled, unroll requested, runtime unroll disabled
+; (runtime unroll disabled to prevent the unroller from succeeding on this
+; runtime-trip-count loop, so the hint survives to transform-warning)
+!10 = distinct !{!10, !11, !12, !13}
+!11 = !{!"llvm.loop.vectorize.enable", i1 false}
+!12 = !{!"llvm.loop.unroll.enable"}
+!13 = !{!"llvm.loop.unroll.runtime.disable"}
+
+; Vectorizable loop: just unroll requested (vectorizer will vectorize it and
+; add unroll.runtime.disable to the remainder and epilogue loops)
+!20 = distinct !{!20, !12}
+
+!100 = distinct !DISubprogram(name: "plain_loop", scope: !1, file: !1, line: 1, type: !99, isLocal: false, isDefinition: true, scopeLine: 1, unit: !0)
+!108 = !DILocation(line: 1, column: 1, scope: !100)
+
+!200 = distinct !DISubprogram(name: "vectorizable_loop", scope: !1, file: !1, line: 10, type: !99, isLocal: false, isDefinition: true, scopeLine: 10, unit: !0)
+!208 = !DILocation(line: 10, column: 1, scope: !200)
diff --git a/llvm/test/Transforms/LoopUnroll/vectorizer-loop-kind-remarks.ll b/llvm/test/Transforms/LoopUnroll/vectorizer-loop-kind-remarks.ll
new file mode 100644
index 0000000000000..723a0273e90a7
--- /dev/null
+++ b/llvm/test/Transforms/LoopUnroll/vectorizer-loop-kind-remarks.ll
@@ -0,0 +1,110 @@
+; RUN: opt < %s -S -passes='loop-vectorize,loop-unroll' \
+; RUN: -pass-remarks=loop-unroll \
+; RUN: -force-vector-width=4 -force-vector-interleave=1 \
+; RUN: -enable-epilogue-vectorization -epilogue-vectorization-force-VF=4 \
+; RUN: -unroll-count=2 -disable-output 2>&1 | FileCheck %s
+; RUN: opt < %s -S -passes='loop-vectorize,loop-unroll' \
+; RUN: -pass-remarks=loop-unroll \
+; RUN: -force-vector-width=4 -force-vector-interleave=1 \
+; RUN: -enable-epilogue-vectorization -epilogue-vectorization-force-VF=4 \
+; RUN: -unroll-count=2 -disable-output -pass-remarks-output=%t.yaml 2>&1
+; RUN: FileCheck --input-file=%t.yaml %s --check-prefix=YAML
+
+; Verify that when the loop vectorizer produces body / epilogue
+; metadata and the unroller successfully unrolls the resulting loops, each
+; unroll remark carries the correct loop-kind qualifier.
+;
+; Pipeline: loop-vectorize -> loop-unroll (with forced unroll count).
+; Epilogue vectorization is forced to exercise all four loop categories:
+;
+; 1. plain loop – not touched by the vectorizer
+; 2. vectorized loop – main vector body (has vectorize.body)
+; 3. epilogue loop – epilogue loop (has vectorize.epilogue)
+;
+; Both stderr remarks and YAML structured output are checked.
+
+;--- plain_loop: vectorize.enable=false keeps this a plain scalar loop --------
+
+; CHECK: remark: test.cpp:1:1: unrolled loop by a factor of 2
+; CHECK-NOT: remark: test.cpp:1:1: unrolled vectorized
+; CHECK-NOT: remark: test.cpp:1:1: unrolled epilogue
+
+define void @plain_loop(ptr noalias %A, i64 %n) !dbg !100 {
+entry:
+ br label %loop, !dbg !108
+loop:
+ %i = phi i64 [0, %entry], [%i.next, %loop]
+ %gep = getelementptr inbounds i32, ptr %A, i64 %i
+ %v = load i32, ptr %gep, align 4
+ %add = add i32 %v, 1
+ store i32 %add, ptr %gep, align 4
+ %i.next = add nuw nsw i64 %i, 1
+ %cmp = icmp slt i64 %i.next, %n
+ br i1 %cmp, label %loop, label %exit, !dbg !108, !llvm.loop !10
+exit:
+ ret void
+}
+
+;--- vectorizable_loop: vectorized with epilogue -> 3 unrolled sub-loops ------
+
+; CHECK-DAG: remark: test.cpp:10:1: unrolled vectorized loop by a factor of 2
+; CHECK-DAG: remark: test.cpp:10:1: unrolled epilogue loop by a factor of 2
+
+define void @vectorizable_loop(ptr noalias %A, ptr noalias %B, i64 %n) !dbg !200 {
+entry:
+ br label %loop, !dbg !208
+loop:
+ %i = phi i64 [0, %entry], [%i.next, %loop]
+ %gep.a = getelementptr inbounds i32, ptr %A, i64 %i
+ %gep.b = getelementptr inbounds i32, ptr %B, i64 %i
+ %v = load i32, ptr %gep.a, align 4
+ %add = add i32 %v, 1
+ store i32 %add, ptr %gep.b, align 4
+ %i.next = add nuw nsw i64 %i, 1
+ %cmp = icmp slt i64 %i.next, %n
+ br i1 %cmp, label %loop, label %exit, !dbg !208
+exit:
+ ret void
+}
+
+;--- YAML checks ---------------------------------------------------------------
+
+; YAML: --- !Passed
+; YAML: Pass: loop-unroll
+; YAML: Name: PartialUnrolled
+; YAML: Function: plain_loop
+; YAML: Args:
+; YAML: - String: 'unrolled loop by a factor of '
+
+; YAML: --- !Passed
+; YAML: Pass: loop-unroll
+; YAML: Name: PartialUnrolled
+; YAML: Function: vectorizable_loop
+; YAML: Args:
+; YAML: - String: 'unrolled vectorized loop by a factor of '
+
+; YAML: --- !Passed
+; YAML: Pass: loop-unroll
+; YAML: Name: PartialUnrolled
+; YAML: Function: vectorizable_loop
+; YAML: Args:
+; YAML: - String: 'unrolled epilogue loop by a factor of '
+
+;--- Metadata ------------------------------------------------------------------
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!2, !3}
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: NoDebug)
+!1 = !DIFile(filename: "test.cpp", directory: ".")
+!2 = !{i32 2, !"Dwarf Version", i32 4}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!99 = !DISubroutineType(types: !{})
+
+!10 = distinct !{!10, !11}
+!11 = !{!"llvm.loop.vectorize.enable", i1 false}
+
+!100 = distinct !DISubprogram(name: "plain_loop", scope: !1, file: !1, line: 1, type: !99, isLocal: false, isDefinition: true, scopeLine: 1, unit: !0)
+!108 = !DILocation(line: 1, column: 1, scope: !100)
+
+!200 = distinct !DISubprogram(name: "vectorizable_loop", scope: !1, file: !1, line: 10, type: !99, isLocal: false, isDefinition: true, scopeLine: 10, unit: !0)
+!208 = !DILocation(line: 10, column: 1, scope: !200)
diff --git a/llvm/test/Transforms/LoopVectorize/RISCV/remark-reductions.ll b/llvm/test/Transforms/LoopVectorize/RISCV/remark-reductions.ll
index b80368df96089..e6bb4b9c200c2 100644
--- a/llvm/test/Transforms/LoopVectorize/RISCV/remark-reductions.ll
+++ b/llvm/test/Transforms/LoopVectorize/RISCV/remark-reductions.ll
@@ -40,7 +40,8 @@ exit:
ret float %red.lcssa
}
;.
-; CHECK: [[LOOP0]] = distinct !{[[LOOP0]], [[META1:![0-9]+]], [[META2:![0-9]+]]}
+; CHECK: [[LOOP0]] = distinct !{[[LOOP0]], [[META1:![0-9]+]], [[META2:![0-9]+]], [[META3:![0-9]+]]}
; CHECK: [[META1]] = !{!"llvm.loop.isvectorized", i32 1}
-; CHECK: [[META2]] = !{!"llvm.loop.unroll.runtime.disable"}
+; CHECK: [[META2]] = !{!"llvm.loop.vectorize.body", i32 1}
+; CHECK: [[META3]] = !{!"llvm.loop.unroll.runtime.disable"}
;.
diff --git a/llvm/test/Transforms/LoopVectorize/remarks-reduction-inloop.ll b/llvm/test/Transforms/LoopVectorize/remarks-reduction-inloop.ll
index 94f9734d425d6..ece9735903608 100644
--- a/llvm/test/Transforms/LoopVectorize/remarks-reduction-inloop.ll
+++ b/llvm/test/Transforms/LoopVectorize/remarks-reduction-inloop.ll
@@ -45,7 +45,8 @@ exit:
ret i32 %sum.0.lcssa
}
;.
-; CHECK: [[LOOP0]] = distinct !{[[LOOP0]], [[META1:![0-9]+]], [[META2:![0-9]+]]}
+; CHECK: [[LOOP0]] = distinct !{[[LOOP0]], [[META1:![0-9]+]], [[META2:![0-9]+]], [[META3:![0-9]+]]}
; CHECK: [[META1]] = !{!"llvm.loop.isvectorized", i32 1}
-; CHECK: [[META2]] = !{!"llvm.loop.unroll.runtime.disable"}
+; CHECK: [[META2]] = !{!"llvm.loop.vectorize.body", i32 1}
+; CHECK: [[META3]] = !{!"llvm.loop.unroll.runtime.disable"}
;.
diff --git a/llvm/test/Transforms/LoopVectorize/vectorize-loop-kind-metadata.ll b/llvm/test/Transforms/LoopVectorize/vectorize-loop-kind-metadata.ll
new file mode 100644
index 0000000000000..c79ae619cd51c
--- /dev/null
+++ b/llvm/test/Transforms/LoopVectorize/vectorize-loop-kind-metadata.ll
@@ -0,0 +1,34 @@
+; RUN: opt < %s -passes=loop-vectorize -force-vector-interleave=1 -force-vector-width=4 \
+; RUN: -pass-remarks-analysis=loop-vectorize -S | FileCheck %s --check-prefix=REMARKS
+; RUN: opt < %s -passes=loop-vectorize -force-vector-interleave=1 -force-vector-width=4 -S \
+; RUN: | FileCheck %s --check-prefix=NO-REMARKS
+
+; Verify that llvm.loop.vectorize.body is attached to the vector loop and
+; llvm.loop.vectorize.epilogue is attached to the scalar remainder when
+; optimization remarks are enabled, and absent otherwise.
+
+define void @test_metadata(ptr noalias %A, i64 %n) {
+entry:
+ br label %loop
+
+loop:
+ %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
+ %gep = getelementptr inbounds i32, ptr %A, i64 %iv
+ %val = load i32, ptr %gep, align 4
+ %add = add i32 %val, 1
+ store i32 %add, ptr %gep, align 4
+ %iv.next = add i64 %iv, 1
+ %cmp = icmp slt i64 %iv.next, %n
+ br i1 %cmp, label %loop, label %exit
+
+exit:
+ ret void
+}
+
+; With remarks enabled: vector loop has vectorize.body, scalar remainder has vectorize.epilogue.
+; REMARKS-DAG: !{!"llvm.loop.vectorize.body", i32 1}
+; REMARKS-DAG: !{!"llvm.loop.vectorize.epilogue", i32 1}
+
+; Without remarks: neither metadata is present.
+; NO-REMARKS-NOT: llvm.loop.vectorize.body
+; NO-REMARKS-NOT: llvm.loop.vectorize.epilogue
More information about the llvm-commits
mailing list