[llvm-branch-commits] [clang] 38d18d9 - [SVE] Add support to vectorize_width loop pragma for scalable vectors
David Sherwood via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Fri Jan 8 04:03:56 PST 2021
Author: David Sherwood
Date: 2021-01-08T11:37:27Z
New Revision: 38d18d93534d290d045bbbfa86337e70f1139dc2
URL: https://github.com/llvm/llvm-project/commit/38d18d93534d290d045bbbfa86337e70f1139dc2
DIFF: https://github.com/llvm/llvm-project/commit/38d18d93534d290d045bbbfa86337e70f1139dc2.diff
LOG: [SVE] Add support to vectorize_width loop pragma for scalable vectors
This patch adds support for two new variants of the vectorize_width
pragma:
1. vectorize_width(X[, fixed|scalable]) where an optional second
parameter is passed to the vectorize_width pragma, which indicates if
the user wishes to use fixed width or scalable vectorization. For
example the user can now write something like:
#pragma clang loop vectorize_width(4, fixed)
or
#pragma clang loop vectorize_width(4, scalable)
In the absence of a second parameter it is assumed the user wants
fixed width vectorization, in order to maintain compatibility with
existing code.
2. vectorize_width(fixed|scalable) where the width is left unspecified,
but the user hints what type of vectorization they prefer, either
fixed width or scalable.
I have implemented this by making use of the LLVM loop hint attribute:
llvm.loop.vectorize.scalable.enable
Tests were added to
clang/test/CodeGenCXX/pragma-loop.cpp
for both the 'fixed' and 'scalable' optional parameter.
See this thread for context: http://lists.llvm.org/pipermail/cfe-dev/2020-November/067262.html
Differential Revision: https://reviews.llvm.org/D89031
Added:
Modified:
clang/docs/LanguageExtensions.rst
clang/include/clang/Basic/Attr.td
clang/include/clang/Basic/DiagnosticParseKinds.td
clang/lib/AST/AttrImpl.cpp
clang/lib/CodeGen/CGLoopInfo.cpp
clang/lib/CodeGen/CGLoopInfo.h
clang/lib/Parse/ParsePragma.cpp
clang/lib/Sema/SemaStmtAttr.cpp
clang/test/AST/ast-print-pragmas.cpp
clang/test/CodeGenCXX/pragma-loop-pr27643.cpp
clang/test/CodeGenCXX/pragma-loop.cpp
clang/test/Parser/pragma-loop.cpp
Removed:
################################################################################
diff --git a/clang/docs/LanguageExtensions.rst b/clang/docs/LanguageExtensions.rst
index 0c01a2bbc52b..6fa6c55b15fc 100644
--- a/clang/docs/LanguageExtensions.rst
+++ b/clang/docs/LanguageExtensions.rst
@@ -3107,8 +3107,18 @@ manually enable vectorization or interleaving.
...
}
-The vector width is specified by ``vectorize_width(_value_)`` and the interleave
-count is specified by ``interleave_count(_value_)``, where
+The vector width is specified by
+``vectorize_width(_value_[, fixed|scalable])``, where _value_ is a positive
+integer and the type of vectorization can be specified with an optional
+second parameter. The default for the second parameter is 'fixed' and
+refers to fixed width vectorization, whereas 'scalable' indicates the
+compiler should use scalable vectors instead. Another use of vectorize_width
+is ``vectorize_width(fixed|scalable)`` where the user can hint at the type
+of vectorization to use without specifying the exact width. In both variants
+of the pragma the vectorizer may decide to fall back on fixed width
+vectorization if the target does not support scalable vectors.
+
+The interleave count is specified by ``interleave_count(_value_)``, where
_value_ is a positive integer. This is useful for specifying the optimal
width/count of the set of target architectures supported by your application.
diff --git a/clang/include/clang/Basic/Attr.td b/clang/include/clang/Basic/Attr.td
index b84e6a14f371..248409946123 100644
--- a/clang/include/clang/Basic/Attr.td
+++ b/clang/include/clang/Basic/Attr.td
@@ -3375,8 +3375,10 @@ def LoopHint : Attr {
"PipelineDisabled", "PipelineInitiationInterval", "Distribute",
"VectorizePredicate"]>,
EnumArgument<"State", "LoopHintState",
- ["enable", "disable", "numeric", "assume_safety", "full"],
- ["Enable", "Disable", "Numeric", "AssumeSafety", "Full"]>,
+ ["enable", "disable", "numeric", "fixed_width",
+ "scalable_width", "assume_safety", "full"],
+ ["Enable", "Disable", "Numeric", "FixedWidth",
+ "ScalableWidth", "AssumeSafety", "Full"]>,
ExprArgument<"Value">];
let AdditionalMembers = [{
diff --git a/clang/include/clang/Basic/DiagnosticParseKinds.td b/clang/include/clang/Basic/DiagnosticParseKinds.td
index 8f78bbfc4e70..0ed80a481e78 100644
--- a/clang/include/clang/Basic/DiagnosticParseKinds.td
+++ b/clang/include/clang/Basic/DiagnosticParseKinds.td
@@ -1396,6 +1396,12 @@ def err_pragma_loop_invalid_option : Error<
"%select{invalid|missing}0 option%select{ %1|}0; expected vectorize, "
"vectorize_width, interleave, interleave_count, unroll, unroll_count, "
"pipeline, pipeline_initiation_interval, vectorize_predicate, or distribute">;
+def err_pragma_loop_invalid_vectorize_option : Error<
+ "vectorize_width loop hint malformed; use vectorize_width(X, fixed) or "
+ "vectorize_width(X, scalable) where X is an integer, or vectorize_width('fixed' or 'scalable')">;
+def note_pragma_loop_invalid_vectorize_option : Note<
+ "vectorize_width loop hint malformed; use vectorize_width(X, fixed) or "
+ "vectorize_width(X, scalable) where X is an integer, or vectorize_width('fixed' or 'scalable')">;
def err_pragma_fp_invalid_option : Error<
"%select{invalid|missing}0 option%select{ %1|}0; expected 'contract', 'reassociate' or 'exceptions'">;
diff --git a/clang/lib/AST/AttrImpl.cpp b/clang/lib/AST/AttrImpl.cpp
index 7ca58f2b83a2..09fdca67995f 100644
--- a/clang/lib/AST/AttrImpl.cpp
+++ b/clang/lib/AST/AttrImpl.cpp
@@ -42,7 +42,16 @@ std::string LoopHintAttr::getValueString(const PrintingPolicy &Policy) const {
OS << "(";
if (state == Numeric)
value->printPretty(OS, nullptr, Policy);
- else if (state == Enable)
+ else if (state == FixedWidth || state == ScalableWidth) {
+ if (value) {
+ value->printPretty(OS, nullptr, Policy);
+ if (state == ScalableWidth)
+ OS << ", scalable";
+ } else if (state == ScalableWidth)
+ OS << "scalable";
+ else
+ OS << "fixed";
+ } else if (state == Enable)
OS << "enable";
else if (state == Full)
OS << "full";
diff --git a/clang/lib/CodeGen/CGLoopInfo.cpp b/clang/lib/CodeGen/CGLoopInfo.cpp
index be4993e802f8..8ba40599cfaf 100644
--- a/clang/lib/CodeGen/CGLoopInfo.cpp
+++ b/clang/lib/CodeGen/CGLoopInfo.cpp
@@ -217,7 +217,8 @@ LoopInfo::createLoopVectorizeMetadata(const LoopAttributes &Attrs,
Enabled = false;
else if (Attrs.VectorizeEnable != LoopAttributes::Unspecified ||
Attrs.VectorizePredicateEnable != LoopAttributes::Unspecified ||
- Attrs.InterleaveCount != 0 || Attrs.VectorizeWidth != 0)
+ Attrs.InterleaveCount != 0 || Attrs.VectorizeWidth != 0 ||
+ Attrs.VectorizeScalable != LoopAttributes::Unspecified)
Enabled = true;
if (Enabled != true) {
@@ -271,6 +272,16 @@ LoopInfo::createLoopVectorizeMetadata(const LoopAttributes &Attrs,
MDString::get(Ctx, "llvm.loop.vectorize.width"),
ConstantAsMetadata::get(ConstantInt::get(llvm::Type::getInt32Ty(Ctx),
Attrs.VectorizeWidth))};
+
+ Args.push_back(MDNode::get(Ctx, Vals));
+ }
+
+ if (Attrs.VectorizeScalable != LoopAttributes::Unspecified) {
+ bool IsScalable = Attrs.VectorizeScalable == LoopAttributes::Enable;
+ Metadata *Vals[] = {
+ MDString::get(Ctx, "llvm.loop.vectorize.scalable.enable"),
+ ConstantAsMetadata::get(
+ ConstantInt::get(llvm::Type::getInt1Ty(Ctx), IsScalable))};
Args.push_back(MDNode::get(Ctx, Vals));
}
@@ -286,10 +297,16 @@ LoopInfo::createLoopVectorizeMetadata(const LoopAttributes &Attrs,
// vectorize.enable is set if:
// 1) loop hint vectorize.enable is set, or
// 2) it is implied when vectorize.predicate is set, or
- // 3) it is implied when vectorize.width is set.
+ // 3) it is implied when vectorize.width is set to a value > 1
+ // 4) it is implied when vectorize.scalable.enable is true
+ // 5) it is implied when vectorize.width is unset (0) and the user
+ // explicitly requested fixed-width vectorization, i.e.
+ // vectorize.scalable.enable is false.
if (Attrs.VectorizeEnable != LoopAttributes::Unspecified ||
- IsVectorPredicateEnabled ||
- Attrs.VectorizeWidth > 1 ) {
+ IsVectorPredicateEnabled || Attrs.VectorizeWidth > 1 ||
+ Attrs.VectorizeScalable == LoopAttributes::Enable ||
+ (Attrs.VectorizeScalable == LoopAttributes::Disable &&
+ Attrs.VectorizeWidth != 1)) {
bool AttrVal = Attrs.VectorizeEnable != LoopAttributes::Disable;
Args.push_back(
MDNode::get(Ctx, {MDString::get(Ctx, "llvm.loop.vectorize.enable"),
@@ -433,13 +450,15 @@ LoopAttributes::LoopAttributes(bool IsParallel)
UnrollEnable(LoopAttributes::Unspecified),
UnrollAndJamEnable(LoopAttributes::Unspecified),
VectorizePredicateEnable(LoopAttributes::Unspecified), VectorizeWidth(0),
- InterleaveCount(0), UnrollCount(0), UnrollAndJamCount(0),
+ VectorizeScalable(LoopAttributes::Unspecified), InterleaveCount(0),
+ UnrollCount(0), UnrollAndJamCount(0),
DistributeEnable(LoopAttributes::Unspecified), PipelineDisabled(false),
PipelineInitiationInterval(0), MustProgress(false) {}
void LoopAttributes::clear() {
IsParallel = false;
VectorizeWidth = 0;
+ VectorizeScalable = LoopAttributes::Unspecified;
InterleaveCount = 0;
UnrollCount = 0;
UnrollAndJamCount = 0;
@@ -466,6 +485,7 @@ LoopInfo::LoopInfo(BasicBlock *Header, const LoopAttributes &Attrs,
}
if (!Attrs.IsParallel && Attrs.VectorizeWidth == 0 &&
+ Attrs.VectorizeScalable == LoopAttributes::Unspecified &&
Attrs.InterleaveCount == 0 && Attrs.UnrollCount == 0 &&
Attrs.UnrollAndJamCount == 0 && !Attrs.PipelineDisabled &&
Attrs.PipelineInitiationInterval == 0 &&
@@ -501,6 +521,7 @@ void LoopInfo::finish() {
BeforeJam.IsParallel = AfterJam.IsParallel = Attrs.IsParallel;
BeforeJam.VectorizeWidth = Attrs.VectorizeWidth;
+ BeforeJam.VectorizeScalable = Attrs.VectorizeScalable;
BeforeJam.InterleaveCount = Attrs.InterleaveCount;
BeforeJam.VectorizeEnable = Attrs.VectorizeEnable;
BeforeJam.DistributeEnable = Attrs.DistributeEnable;
@@ -543,7 +564,8 @@ void LoopInfo::finish() {
SmallVector<Metadata *, 1> BeforeLoopProperties;
if (BeforeJam.VectorizeEnable != LoopAttributes::Unspecified ||
BeforeJam.VectorizePredicateEnable != LoopAttributes::Unspecified ||
- BeforeJam.InterleaveCount != 0 || BeforeJam.VectorizeWidth != 0)
+ BeforeJam.InterleaveCount != 0 || BeforeJam.VectorizeWidth != 0 ||
+ BeforeJam.VectorizeScalable == LoopAttributes::Enable)
BeforeLoopProperties.push_back(
MDNode::get(Ctx, MDString::get(Ctx, "llvm.loop.isvectorized")));
@@ -620,6 +642,7 @@ void LoopInfoStack::push(BasicBlock *Header, clang::ASTContext &Ctx,
case LoopHintAttr::Vectorize:
// Disable vectorization by specifying a width of 1.
setVectorizeWidth(1);
+ setVectorizeScalable(LoopAttributes::Unspecified);
break;
case LoopHintAttr::Interleave:
// Disable interleaving by speciyfing a count of 1.
@@ -721,11 +744,23 @@ void LoopInfoStack::push(BasicBlock *Header, clang::ASTContext &Ctx,
break;
}
break;
- case LoopHintAttr::Numeric:
+ case LoopHintAttr::FixedWidth:
+ case LoopHintAttr::ScalableWidth:
switch (Option) {
case LoopHintAttr::VectorizeWidth:
- setVectorizeWidth(ValueInt);
+ setVectorizeScalable(State == LoopHintAttr::ScalableWidth
+ ? LoopAttributes::Enable
+ : LoopAttributes::Disable);
+ if (LH->getValue())
+ setVectorizeWidth(ValueInt);
break;
+ default:
+ llvm_unreachable("Options cannot be used with 'scalable' hint.");
+ break;
+ }
+ break;
+ case LoopHintAttr::Numeric:
+ switch (Option) {
case LoopHintAttr::InterleaveCount:
setInterleaveCount(ValueInt);
break;
@@ -742,6 +777,7 @@ void LoopInfoStack::push(BasicBlock *Header, clang::ASTContext &Ctx,
case LoopHintAttr::UnrollAndJam:
case LoopHintAttr::VectorizePredicate:
case LoopHintAttr::Vectorize:
+ case LoopHintAttr::VectorizeWidth:
case LoopHintAttr::Interleave:
case LoopHintAttr::Distribute:
case LoopHintAttr::PipelineDisabled:
diff --git a/clang/lib/CodeGen/CGLoopInfo.h b/clang/lib/CodeGen/CGLoopInfo.h
index 2c13e020c5df..856e892f712e 100644
--- a/clang/lib/CodeGen/CGLoopInfo.h
+++ b/clang/lib/CodeGen/CGLoopInfo.h
@@ -58,6 +58,9 @@ struct LoopAttributes {
/// Value for llvm.loop.vectorize.width metadata.
unsigned VectorizeWidth;
+ // Value for llvm.loop.vectorize.scalable.enable
+ LVEnableState VectorizeScalable;
+
/// Value for llvm.loop.interleave.count metadata.
unsigned InterleaveCount;
@@ -258,6 +261,10 @@ class LoopInfoStack {
/// Set the vectorize width for the next loop pushed.
void setVectorizeWidth(unsigned W) { StagedAttrs.VectorizeWidth = W; }
+ void setVectorizeScalable(const LoopAttributes::LVEnableState &State) {
+ StagedAttrs.VectorizeScalable = State;
+ }
+
/// Set the interleave count for the next loop pushed.
void setInterleaveCount(unsigned C) { StagedAttrs.InterleaveCount = C; }
diff --git a/clang/lib/Parse/ParsePragma.cpp b/clang/lib/Parse/ParsePragma.cpp
index 64de2cf5d7f1..8c586b3eaf6c 100644
--- a/clang/lib/Parse/ParsePragma.cpp
+++ b/clang/lib/Parse/ParsePragma.cpp
@@ -1187,12 +1187,79 @@ bool Parser::HandlePragmaLoopHint(LoopHint &Hint) {
Diag(Tok.getLocation(), diag::warn_pragma_extra_tokens_at_eol)
<< PragmaLoopHintString(Info->PragmaName, Info->Option);
Hint.StateLoc = IdentifierLoc::create(Actions.Context, StateLoc, StateInfo);
+ } else if (OptionInfo && OptionInfo->getName() == "vectorize_width") {
+ PP.EnterTokenStream(Toks, /*DisableMacroExpansion=*/false,
+ /*IsReinject=*/false);
+ ConsumeAnnotationToken();
+
+ SourceLocation StateLoc = Toks[0].getLocation();
+ IdentifierInfo *StateInfo = Toks[0].getIdentifierInfo();
+ StringRef IsScalableStr = StateInfo ? StateInfo->getName() : "";
+
+ // Look for vectorize_width(fixed|scalable)
+ if (IsScalableStr == "scalable" || IsScalableStr == "fixed") {
+ PP.Lex(Tok); // Identifier
+
+ if (Toks.size() > 2) {
+ Diag(Tok.getLocation(), diag::warn_pragma_extra_tokens_at_eol)
+ << PragmaLoopHintString(Info->PragmaName, Info->Option);
+ while (Tok.isNot(tok::eof))
+ ConsumeAnyToken();
+ }
+
+ Hint.StateLoc =
+ IdentifierLoc::create(Actions.Context, StateLoc, StateInfo);
+
+ ConsumeToken(); // Consume the constant expression eof terminator.
+ } else {
+ // Enter constant expression including eof terminator into token stream.
+ ExprResult R = ParseConstantExpression();
+
+ if (R.isInvalid() && !Tok.is(tok::comma))
+ Diag(Toks[0].getLocation(),
+ diag::note_pragma_loop_invalid_vectorize_option);
+
+ bool Arg2Error = false;
+ if (Tok.is(tok::comma)) {
+ PP.Lex(Tok); // ,
+
+ StateInfo = Tok.getIdentifierInfo();
+ IsScalableStr = StateInfo->getName();
+
+ if (IsScalableStr != "scalable" && IsScalableStr != "fixed") {
+ Diag(Tok.getLocation(),
+ diag::err_pragma_loop_invalid_vectorize_option);
+ Arg2Error = true;
+ } else
+ Hint.StateLoc =
+ IdentifierLoc::create(Actions.Context, StateLoc, StateInfo);
+
+ PP.Lex(Tok); // Identifier
+ }
+
+ // Tokens following an error in an ill-formed constant expression will
+ // remain in the token stream and must be removed.
+ if (Tok.isNot(tok::eof)) {
+ Diag(Tok.getLocation(), diag::warn_pragma_extra_tokens_at_eol)
+ << PragmaLoopHintString(Info->PragmaName, Info->Option);
+ while (Tok.isNot(tok::eof))
+ ConsumeAnyToken();
+ }
+
+ ConsumeToken(); // Consume the constant expression eof terminator.
+
+ if (Arg2Error || R.isInvalid() ||
+ Actions.CheckLoopHintExpr(R.get(), Toks[0].getLocation()))
+ return false;
+
+ // Argument is a constant expression with an integer type.
+ Hint.ValueExpr = R.get();
+ }
} else {
// Enter constant expression including eof terminator into token stream.
PP.EnterTokenStream(Toks, /*DisableMacroExpansion=*/false,
/*IsReinject=*/false);
ConsumeAnnotationToken();
-
ExprResult R = ParseConstantExpression();
// Tokens following an error in an ill-formed constant expression will
diff --git a/clang/lib/Sema/SemaStmtAttr.cpp b/clang/lib/Sema/SemaStmtAttr.cpp
index 58f4d0dc1944..8031aa6b0ece 100644
--- a/clang/lib/Sema/SemaStmtAttr.cpp
+++ b/clang/lib/Sema/SemaStmtAttr.cpp
@@ -10,13 +10,14 @@
//
//===----------------------------------------------------------------------===//
-#include "clang/AST/EvaluatedExprVisitor.h"
-#include "clang/Sema/SemaInternal.h"
#include "clang/AST/ASTContext.h"
+#include "clang/AST/EvaluatedExprVisitor.h"
#include "clang/Basic/SourceManager.h"
+#include "clang/Basic/TargetInfo.h"
#include "clang/Sema/DelayedDiagnostic.h"
#include "clang/Sema/Lookup.h"
#include "clang/Sema/ScopeInfo.h"
+#include "clang/Sema/SemaInternal.h"
#include "llvm/ADT/StringExtras.h"
using namespace clang;
@@ -139,10 +140,18 @@ static Attr *handleLoopHintAttr(Sema &S, Stmt *St, const ParsedAttr &A,
LoopHintAttr::PipelineInitiationInterval)
.Case("distribute", LoopHintAttr::Distribute)
.Default(LoopHintAttr::Vectorize);
- if (Option == LoopHintAttr::VectorizeWidth ||
- Option == LoopHintAttr::InterleaveCount ||
- Option == LoopHintAttr::UnrollCount ||
- Option == LoopHintAttr::PipelineInitiationInterval) {
+ if (Option == LoopHintAttr::VectorizeWidth) {
+ assert((ValueExpr || (StateLoc && StateLoc->Ident)) &&
+ "Attribute must have a valid value expression or argument.");
+ if (ValueExpr && S.CheckLoopHintExpr(ValueExpr, St->getBeginLoc()))
+ return nullptr;
+ if (StateLoc && StateLoc->Ident && StateLoc->Ident->isStr("scalable"))
+ State = LoopHintAttr::ScalableWidth;
+ else
+ State = LoopHintAttr::FixedWidth;
+ } else if (Option == LoopHintAttr::InterleaveCount ||
+ Option == LoopHintAttr::UnrollCount ||
+ Option == LoopHintAttr::PipelineInitiationInterval) {
assert(ValueExpr && "Attribute must have a valid value expression.");
if (S.CheckLoopHintExpr(ValueExpr, St->getBeginLoc()))
return nullptr;
diff --git a/clang/test/AST/ast-print-pragmas.cpp b/clang/test/AST/ast-print-pragmas.cpp
index 7bd0b48d6e77..5f27ff71b4ee 100644
--- a/clang/test/AST/ast-print-pragmas.cpp
+++ b/clang/test/AST/ast-print-pragmas.cpp
@@ -8,6 +8,36 @@ void test(int *List, int Length) {
int i = 0;
#pragma clang loop vectorize_width(4)
#pragma clang loop interleave_count(8)
+// CHECK-NEXT: while (i < Length)
+ while (i < Length) {
+ List[i] = i * 2;
+ i++;
+ }
+ i = 0;
+
+// CHECK: #pragma clang loop vectorize_width(4, scalable)
+
+#pragma clang loop vectorize_width(4, scalable)
+// CHECK-NEXT: while (i < Length)
+ while (i < Length) {
+ List[i] = i * 2;
+ i++;
+ }
+ i = 0;
+
+// CHECK: #pragma clang loop vectorize_width(fixed)
+
+#pragma clang loop vectorize_width(fixed)
+// CHECK-NEXT: while (i < Length)
+ while (i < Length) {
+ List[i] = i * 2;
+ i++;
+ }
+ i = 0;
+
+// CHECK: #pragma clang loop vectorize_width(scalable)
+
+#pragma clang loop vectorize_width(scalable)
// CHECK-NEXT: while (i < Length)
while (i < Length) {
List[i] = i * 2;
diff --git a/clang/test/CodeGenCXX/pragma-loop-pr27643.cpp b/clang/test/CodeGenCXX/pragma-loop-pr27643.cpp
index 75c2668962d4..37e29776264b 100644
--- a/clang/test/CodeGenCXX/pragma-loop-pr27643.cpp
+++ b/clang/test/CodeGenCXX/pragma-loop-pr27643.cpp
@@ -40,13 +40,14 @@ void loop4(int *List, int Length) {
List[i] = i * 2;
}
-// CHECK: ![[LOOP1]] = distinct !{![[LOOP1]], [[MP:![0-9]+]], ![[VEC_WIDTH_1:.*]], ![[VEC_ENABLE:.*]]}
+// CHECK: ![[LOOP1]] = distinct !{![[LOOP1]], [[MP:![0-9]+]], ![[VEC_WIDTH_1:.*]], ![[FIXED_WIDTH:.*]], ![[VEC_ENABLE:.*]]}
// CHECK: ![[VEC_WIDTH_1]] = !{!"llvm.loop.vectorize.width", i32 1}
+// CHECK: ![[FIXED_WIDTH]] = !{!"llvm.loop.vectorize.scalable.enable", i1 false}
// CHECK: ![[VEC_ENABLE]] = !{!"llvm.loop.vectorize.enable", i1 true}
-// CHECK: ![[LOOP2]] = distinct !{![[LOOP2]], [[MP]], ![[VEC_WIDTH_2:.*]], ![[VEC_ENABLE]]}
+// CHECK: ![[LOOP2]] = distinct !{![[LOOP2]], [[MP]], ![[VEC_WIDTH_2:.*]], ![[FIXED_WIDTH:.*]], ![[VEC_ENABLE]]}
// CHECK: ![[VEC_WIDTH_2]] = !{!"llvm.loop.vectorize.width", i32 2}
-// CHECK: ![[LOOP3]] = distinct !{![[LOOP3]], [[MP]], ![[VEC_WIDTH_1]]}
+// CHECK: ![[LOOP3]] = distinct !{![[LOOP3]], [[MP]], ![[VEC_WIDTH_1]], ![[FIXED_WIDTH]]}
-// CHECK: ![[LOOP4]] = distinct !{![[LOOP4]], [[MP]], ![[VEC_WIDTH_2]], ![[VEC_ENABLE]]}
+// CHECK: ![[LOOP4]] = distinct !{![[LOOP4]], [[MP]], ![[VEC_WIDTH_2]], ![[FIXED_WIDTH]], ![[VEC_ENABLE]]}
diff --git a/clang/test/CodeGenCXX/pragma-loop.cpp b/clang/test/CodeGenCXX/pragma-loop.cpp
index a34817082281..127df41522a5 100644
--- a/clang/test/CodeGenCXX/pragma-loop.cpp
+++ b/clang/test/CodeGenCXX/pragma-loop.cpp
@@ -158,51 +158,97 @@ void template_test(double *List, int Length) {
for_template_constant_expression_test<double, 2, 4, 8>(List, Length);
}
+// Verify for loop is performing fixed width vectorization
+void for_test_fixed_16(int *List, int Length) {
+#pragma clang loop vectorize_width(16, fixed) interleave_count(4) unroll(disable) distribute(disable)
+ for (int i = 0; i < Length; i++) {
+ // CHECK: br label {{.*}}, !llvm.loop ![[LOOP_15:.*]]
+ List[i] = i * 2;
+ }
+}
+
+// Verify for loop is performing scalable vectorization
+void for_test_scalable_16(int *List, int Length) {
+#pragma clang loop vectorize_width(16, scalable) interleave_count(4) unroll(disable) distribute(disable)
+ for (int i = 0; i < Length; i++) {
+ // CHECK: br label {{.*}}, !llvm.loop ![[LOOP_16:.*]]
+ List[i] = i * 2;
+ }
+}
+
+// Verify for loop is performing fixed width vectorization
+void for_test_fixed(int *List, int Length) {
+#pragma clang loop vectorize_width(fixed) interleave_count(4) unroll(disable) distribute(disable)
+ for (int i = 0; i < Length; i++) {
+ // CHECK: br label {{.*}}, !llvm.loop ![[LOOP_17:.*]]
+ List[i] = i * 2;
+ }
+}
+
+// Verify for loop is performing scalable vectorization
+void for_test_scalable(int *List, int Length) {
+#pragma clang loop vectorize_width(scalable) interleave_count(4) unroll(disable) distribute(disable)
+ for (int i = 0; i < Length; i++) {
+ // CHECK: br label {{.*}}, !llvm.loop ![[LOOP_18:.*]]
+ List[i] = i * 2;
+ }
+}
+
+// Verify for loop is performing scalable vectorization
+void for_test_scalable_1(int *List, int Length) {
+#pragma clang loop vectorize_width(1, scalable) interleave_count(4) unroll(disable) distribute(disable)
+ for (int i = 0; i < Length; i++) {
+ // CHECK: br label {{.*}}, !llvm.loop ![[LOOP_19:.*]]
+ List[i] = i * 2;
+ }
+}
+
// CHECK: ![[LOOP_1]] = distinct !{![[LOOP_1]], [[MP:![0-9]+]], ![[UNROLL_FULL:.*]]}
// CHECK: ![[UNROLL_FULL]] = !{!"llvm.loop.unroll.full"}
-// CHECK: ![[LOOP_2]] = distinct !{![[LOOP_2]], [[MP]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[WIDTH_8:.*]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE:.*]]}
+// CHECK: ![[LOOP_2]] = distinct !{![[LOOP_2]], [[MP]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[WIDTH_8:.*]], ![[FIXED_VEC:.*]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE:.*]]}
// CHECK: ![[UNROLL_DISABLE]] = !{!"llvm.loop.unroll.disable"}
// CHECK: ![[DISTRIBUTE_DISABLE]] = !{!"llvm.loop.distribute.enable", i1 false}
// CHECK: ![[WIDTH_8]] = !{!"llvm.loop.vectorize.width", i32 8}
+// CHECK: ![[FIXED_VEC]] = !{!"llvm.loop.vectorize.scalable.enable", i1 false}
// CHECK: ![[INTERLEAVE_4]] = !{!"llvm.loop.interleave.count", i32 4}
// CHECK: ![[VECTORIZE_ENABLE]] = !{!"llvm.loop.vectorize.enable", i1 true}
-// CHECK: ![[LOOP_3]] = distinct !{![[LOOP_3]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE]], ![[FOLLOWUP_VECTOR_3:.*]]}
+// CHECK: ![[LOOP_3]] = distinct !{![[LOOP_3]], [[MP]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE]], ![[FOLLOWUP_VECTOR_3:.*]]}
// CHECK: ![[FOLLOWUP_VECTOR_3]] = !{!"llvm.loop.vectorize.followup_all", ![[AFTER_VECTOR_3:.*]]}
// CHECK: ![[AFTER_VECTOR_3]] = distinct !{![[AFTER_VECTOR_3]], [[MP]], ![[ISVECTORIZED:.*]], ![[UNROLL_8:.*]]}
// CHECK: ![[ISVECTORIZED]] = !{!"llvm.loop.isvectorized"}
// CHECK: ![[UNROLL_8]] = !{!"llvm.loop.unroll.count", i32 8}
-// CHECK: ![[LOOP_4]] = distinct !{![[LOOP_4]], ![[WIDTH_2:.*]], ![[INTERLEAVE_2:.*]], ![[VECTORIZE_ENABLE]]}
+// CHECK: ![[LOOP_4]] = distinct !{![[LOOP_4]], ![[WIDTH_2:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_2:.*]], ![[VECTORIZE_ENABLE]]}
// CHECK: ![[WIDTH_2]] = !{!"llvm.loop.vectorize.width", i32 2}
// CHECK: ![[INTERLEAVE_2]] = !{!"llvm.loop.interleave.count", i32 2}
// CHECK: ![[LOOP_5]] = distinct !{![[LOOP_5]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[WIDTH_1:.*]]}
// CHECK: ![[WIDTH_1]] = !{!"llvm.loop.vectorize.width", i32 1}
-// CHECK: ![[LOOP_6]] = distinct !{![[LOOP_6]], ![[WIDTH_2:.*]], ![[INTERLEAVE_2:.*]], ![[FOLLOWUP_VECTOR_6:.*]]}
+// CHECK: ![[LOOP_6]] = distinct !{![[LOOP_6]], [[MP]], ![[WIDTH_2:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_2:.*]], ![[FOLLOWUP_VECTOR_6:.*]]}
// CHECK: ![[FOLLOWUP_VECTOR_6]] = !{!"llvm.loop.vectorize.followup_all", ![[AFTER_VECTOR_6:.*]]}
// CHECK: ![[AFTER_VECTOR_6]] = distinct !{![[AFTER_VECTOR_6]], ![[ISVECTORIZED:.*]], ![[UNROLL_8:.*]]}
-// CHECK: ![[LOOP_7]] = distinct !{![[LOOP_7]], [[MP]], ![[WIDTH_5:.*]], ![[VECTORIZE_ENABLE]]}
+// CHECK: ![[LOOP_7]] = distinct !{![[LOOP_7]], [[MP]], ![[WIDTH_5:.*]], ![[FIXED_VEC]], ![[VECTORIZE_ENABLE]]}
// CHECK: ![[WIDTH_5]] = !{!"llvm.loop.vectorize.width", i32 5}
-// CHECK: ![[LOOP_8]] = distinct !{![[LOOP_8]], ![[WIDTH_5:.*]]}
+// CHECK: ![[LOOP_8]] = distinct !{![[LOOP_8]], [[MP]], ![[WIDTH_5:.*]], ![[FIXED_VEC]], ![[VECTORIZE_ENABLE]]}
-// CHECK: ![[LOOP_9]] = distinct !{![[LOOP_9]], ![[WIDTH_8:.*]], ![[INTERLEAVE_8:.*]], ![[FOLLOWUP_VECTOR_9:.*]]}
+// CHECK: ![[LOOP_9]] = distinct !{![[LOOP_9]], ![[WIDTH_8:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_8:.*]], ![[FOLLOWUP_VECTOR_9:.*]]}
// CHECK: ![[FOLLOWUP_VECTOR_9]] = !{!"llvm.loop.vectorize.followup_all", ![[AFTER_VECTOR_9:.*]]}
// CHECK: ![[AFTER_VECTOR_9]] = distinct !{![[AFTER_VECTOR_9]], ![[ISVECTORIZED:.*]], ![[UNROLL_8:.*]]}
-// CHECK: ![[LOOP_10]] = distinct !{![[LOOP_10]], ![[WIDTH_2:.*]], ![[INTERLEAVE_2:.*]], ![[FOLLOWUP_VECTOR_10:.*]]}
+// CHECK: ![[LOOP_10]] = distinct !{![[LOOP_10]], ![[WIDTH_2:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_2:.*]], ![[FOLLOWUP_VECTOR_10:.*]]}
// CHECK: ![[FOLLOWUP_VECTOR_10]] = !{!"llvm.loop.vectorize.followup_all", ![[AFTER_VECTOR_10:.*]]}
// CHECK: ![[AFTER_VECTOR_10]] = distinct !{![[AFTER_VECTOR_10]], ![[ISVECTORIZED:.*]], ![[UNROLL_8:.*]]}
-// CHECK: ![[LOOP_11]] = distinct !{![[LOOP_11]], ![[WIDTH_2:.*]], ![[INTERLEAVE_4:.*]], ![[FOLLOWUP_VECTOR_11:.*]]}
+// CHECK: ![[LOOP_11]] = distinct !{![[LOOP_11]], ![[WIDTH_2:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_4:.*]], ![[FOLLOWUP_VECTOR_11:.*]]}
// CHECK: ![[FOLLOWUP_VECTOR_11]] = !{!"llvm.loop.vectorize.followup_all", ![[AFTER_VECTOR_11:.*]]}
// CHECK: ![[AFTER_VECTOR_11]] = distinct !{![[AFTER_VECTOR_11]], ![[ISVECTORIZED:.*]], ![[UNROLL_8:.*]]}
-// CHECK: ![[LOOP_12]] = distinct !{![[LOOP_12]], ![[WIDTH_6:.*]], ![[INTERLEAVE_10:.*]], ![[FOLLOWUP_VECTOR_12:.*]]}
+// CHECK: ![[LOOP_12]] = distinct !{![[LOOP_12]], ![[WIDTH_6:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_10:.*]], ![[FOLLOWUP_VECTOR_12:.*]]}
// CHECK: ![[FOLLOWUP_VECTOR_12]] = !{!"llvm.loop.vectorize.followup_all", ![[AFTER_VECTOR_12:.*]]}
// CHECK: ![[AFTER_VECTOR_12]] = distinct !{![[AFTER_VECTOR_12]], ![[ISVECTORIZED:.*]], ![[UNROLL_24:.*]]}
// CHECK: ![[UNROLL_24]] = !{!"llvm.loop.unroll.count", i32 24}
@@ -213,5 +259,15 @@ void template_test(double *List, int Length) {
// CHECK: ![[AFTER_VECTOR_13]] = distinct !{![[AFTER_VECTOR_13]], ![[ISVECTORIZED:.*]], ![[UNROLL_32:.*]]}
// CHECK: ![[UNROLL_32]] = !{!"llvm.loop.unroll.count", i32 32}
-// CHECK: ![[LOOP_14]] = distinct !{![[LOOP_14]], [[MP]], ![[WIDTH_10:.*]], ![[VECTORIZE_ENABLE]]}
+// CHECK: ![[LOOP_14]] = distinct !{![[LOOP_14]], [[MP]], ![[WIDTH_10:.*]], ![[FIXED_VEC]], ![[VECTORIZE_ENABLE]]}
// CHECK: ![[WIDTH_10]] = !{!"llvm.loop.vectorize.width", i32 10}
+
+// CHECK: ![[LOOP_15]] = distinct !{![[LOOP_15]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[WIDTH_16:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE:.*]]}
+// CHECK: ![[WIDTH_16]] = !{!"llvm.loop.vectorize.width", i32 16}
+
+// CHECK: ![[LOOP_16]] = distinct !{![[LOOP_16]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[WIDTH_16]], ![[SCALABLE_VEC:.*]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE:.*]]}
+// CHECK: ![[SCALABLE_VEC]] = !{!"llvm.loop.vectorize.scalable.enable", i1 true}
+
+// CHECK: ![[LOOP_17]] = distinct !{![[LOOP_17]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[FIXED_VEC]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE:.*]]}
+// CHECK: ![[LOOP_18]] = distinct !{![[LOOP_18]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[SCALABLE_VEC]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE:.*]]}
+// CHECK: ![[LOOP_19]] = distinct !{![[LOOP_19]], ![[UNROLL_DISABLE:.*]], ![[DISTRIBUTE_DISABLE:.*]], ![[WIDTH_1]], ![[SCALABLE_VEC]], ![[INTERLEAVE_4:.*]], ![[VECTORIZE_ENABLE:.*]]}
diff --git a/clang/test/Parser/pragma-loop.cpp b/clang/test/Parser/pragma-loop.cpp
index 650d21c14826..456b4858f4ec 100644
--- a/clang/test/Parser/pragma-loop.cpp
+++ b/clang/test/Parser/pragma-loop.cpp
@@ -60,7 +60,8 @@ void test_nontype_template_bool(int *List, int Length) {
template <int V, int I>
void test_nontype_template_badarg(int *List, int Length) {
- /* expected-error {{use of undeclared identifier 'Vec'}} */ #pragma clang loop vectorize_width(Vec) interleave_count(I)
+ /* expected-error {{use of undeclared identifier 'Vec'}} */ #pragma clang loop vectorize_width(Vec) interleave_count(I) /*
+ expected-note {{vectorize_width loop hint malformed; use vectorize_width(X, fixed) or vectorize_width(X, scalable) where X is an integer, or vectorize_width('fixed' or 'scalable')}} */
/* expected-error {{use of undeclared identifier 'Int'}} */ #pragma clang loop vectorize_width(V) interleave_count(Int)
for (int i = 0; i < Length; i++) {
List[i] = i;
@@ -74,6 +75,11 @@ void test_type_template_vectorize(int *List, int Length) {
for (int i = 0; i < Length; i++) {
List[i] = i;
}
+
+ /* expected-error {{invalid value '-1'; must be positive}} */ #pragma clang loop vectorize_width(Value, fixed)
+ for (int i = 0; i < Length; i++) {
+ List[i] = i;
+ }
}
void test(int *List, int Length) {
@@ -189,12 +195,15 @@ test_nontype_template_interleave<8>(List, Length);
/* expected-warning {{extra tokens at end of '#pragma clang loop'}} */ #pragma clang loop vectorize_width(1 +) 1
/* expected-warning {{extra tokens at end of '#pragma clang loop'}} */ #pragma clang loop vectorize_width(1) +1
const int VV = 4;
-/* expected-error {{expected expression}} */ #pragma clang loop vectorize_width(VV +/ 2)
-/* expected-error {{use of undeclared identifier 'undefined'}} */ #pragma clang loop vectorize_width(VV+undefined)
+/* expected-error {{expected expression}} */ #pragma clang loop vectorize_width(VV +/ 2) /*
+ expected-note {{vectorize_width loop hint malformed; use vectorize_width(X, fixed) or vectorize_width(X, scalable) where X is an integer, or vectorize_width('fixed' or 'scalable')}} */
+/* expected-error {{use of undeclared identifier 'undefined'}} */ #pragma clang loop vectorize_width(VV+undefined) /*
+ expected-note {{vectorize_width loop hint malformed; use vectorize_width(X, fixed) or vectorize_width(X, scalable) where X is an integer, or vectorize_width('fixed' or 'scalable')}} */
/* expected-error {{expected ')'}} */ #pragma clang loop vectorize_width(1+(^*/2 * ()
/* expected-warning {{extra tokens at end of '#pragma clang loop' - ignored}} */ #pragma clang loop vectorize_width(1+(-0[0]))))))
-/* expected-error {{use of undeclared identifier 'badvalue'}} */ #pragma clang loop vectorize_width(badvalue)
+/* expected-error {{use of undeclared identifier 'badvalue'}} */ #pragma clang loop vectorize_width(badvalue) /*
+ expected-note {{vectorize_width loop hint malformed; use vectorize_width(X, fixed) or vectorize_width(X, scalable) where X is an integer, or vectorize_width('fixed' or 'scalable')}} */
/* expected-error {{use of undeclared identifier 'badvalue'}} */ #pragma clang loop interleave_count(badvalue)
/* expected-error {{use of undeclared identifier 'badvalue'}} */ #pragma clang loop unroll_count(badvalue)
while (i-6 < Length) {
@@ -215,7 +224,7 @@ const int VV = 4;
/* expected-error {{invalid argument; expected 'enable', 'assume_safety' or 'disable'}} */ #pragma clang loop interleave(*)
/* expected-error {{invalid argument; expected 'enable', 'full' or 'disable'}} */ #pragma clang loop unroll(=)
/* expected-error {{invalid argument; expected 'enable' or 'disable'}} */ #pragma clang loop distribute(+)
-/* expected-error {{type name requires a specifier or qualifier}} expected-error {{expected expression}} */ #pragma clang loop vectorize_width(^)
+/* expected-error {{type name requires a specifier or qualifier}} expected-error {{expected expression}} */ #pragma clang loop vectorize_width(^) /* expected-note {{vectorize_width loop hint malformed; use vectorize_width(X, fixed) or vectorize_width(X, scalable) where X is an integer, or vectorize_width('fixed' or 'scalable')}} */
/* expected-error {{expected expression}} expected-error {{expected expression}} */ #pragma clang loop interleave_count(/)
/* expected-error {{expected expression}} expected-error {{expected expression}} */ #pragma clang loop unroll_count(==)
while (i-8 < Length) {
More information about the llvm-branch-commits
mailing list