[clang] 2bcda6b - [Sema, CodeGen] Implement [[likely]] and [[unlikely]] in SwitchStmt
Mark de Wever via cfe-commits
cfe-commits at lists.llvm.org
Sun Oct 18 04:49:14 PDT 2020
Author: Mark de Wever
Date: 2020-10-18T13:48:42+02:00
New Revision: 2bcda6bb2896f0f8daf67343edfc64fb226f3e3f
URL: https://github.com/llvm/llvm-project/commit/2bcda6bb2896f0f8daf67343edfc64fb226f3e3f
DIFF: https://github.com/llvm/llvm-project/commit/2bcda6bb2896f0f8daf67343edfc64fb226f3e3f.diff
LOG: [Sema, CodeGen] Implement [[likely]] and [[unlikely]] in SwitchStmt
This implements the likelihood attribute for the switch statement. Based on the
discussion in D85091 and D86559 it only handles the attribute when placed on
the case labels or the default labels.
It also marks the likelihood attribute as feature complete. There are more QoI
patches in the pipeline.
Differential Revision: https://reviews.llvm.org/D89210
Added:
clang/test/CodeGenCXX/attr-likelihood-switch-branch-weights.cpp
Modified:
clang/docs/ReleaseNotes.rst
clang/include/clang/AST/Stmt.h
clang/include/clang/Basic/Attr.td
clang/include/clang/Basic/AttrDocs.td
clang/lib/AST/Stmt.cpp
clang/lib/CodeGen/CGStmt.cpp
clang/lib/CodeGen/CodeGenFunction.cpp
clang/lib/CodeGen/CodeGenFunction.h
clang/test/Preprocessor/has_attribute.cpp
clang/www/cxx_status.html
Removed:
################################################################################
diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst
index d0ef03acaa4c..540cf9132605 100644
--- a/clang/docs/ReleaseNotes.rst
+++ b/clang/docs/ReleaseNotes.rst
@@ -123,7 +123,9 @@ New Pragmas in Clang
Attribute Changes in Clang
--------------------------
-- ...
+- Added support for the C++20 likelihood attributes ``[[likely]]`` and
+ ``[[unlikely]]``. As an extension they can be used in C++11 and newer.
+ This extension is enabled by default.
Windows Support
---------------
diff --git a/clang/include/clang/AST/Stmt.h b/clang/include/clang/AST/Stmt.h
index 4a6e8182e5a0..c2452f426566 100644
--- a/clang/include/clang/AST/Stmt.h
+++ b/clang/include/clang/AST/Stmt.h
@@ -1177,6 +1177,9 @@ class alignas(void *) Stmt {
static void EnableStatistics();
static void PrintStats();
+ /// \returns the likelihood of a set of attributes.
+ static Likelihood getLikelihood(ArrayRef<const Attr *> Attrs);
+
/// \returns the likelihood of a statement.
static Likelihood getLikelihood(const Stmt *S);
diff --git a/clang/include/clang/Basic/Attr.td b/clang/include/clang/Basic/Attr.td
index 60e7a9d4303b..f58870d76013 100644
--- a/clang/include/clang/Basic/Attr.td
+++ b/clang/include/clang/Basic/Attr.td
@@ -1290,14 +1290,12 @@ def FallThrough : StmtAttr {
}
def Likely : StmtAttr {
- // FIXME: Change the date to 201803 once the implementation is finished.
- let Spellings = [CXX11<"", "likely", 2>, C2x<"clang", "likely">];
+ let Spellings = [CXX11<"", "likely", 201803>, C2x<"clang", "likely">];
let Documentation = [LikelihoodDocs];
}
def Unlikely : StmtAttr {
- // FIXME: Change the date to 201803 once the implementation is finished.
- let Spellings = [CXX11<"", "unlikely", 2>, C2x<"clang", "unlikely">];
+ let Spellings = [CXX11<"", "unlikely", 201803>, C2x<"clang", "unlikely">];
let Documentation = [LikelihoodDocs];
}
diff --git a/clang/include/clang/Basic/AttrDocs.td b/clang/include/clang/Basic/AttrDocs.td
index bf190829381c..babb7cd05a80 100644
--- a/clang/include/clang/Basic/AttrDocs.td
+++ b/clang/include/clang/Basic/AttrDocs.td
@@ -1724,13 +1724,23 @@ It isn't allowed to annotate a single statement with both ``likely`` and
statement with the same likelihood attribute will result in a diagnostic and
the attributes are ignored on both branches.
+In a ``switch`` statement it's allowed to annotate multiple ``case`` labels
+or the ``default`` label with the same likelihood attribute. This makes
+* all labels without an attribute have a neutral likelihood,
+* all labels marked ``[[likely]]`` have an equally positive likelihood, and
+* all labels marked ``[[unlikely]]`` have an equally negative likelihood.
+The neutral likelihood is the more likely of path execution than the negative
+likelihood. The positive likelihood is the more likely of path of execution
+than the neutral likelihood.
+
These attributes have no effect on the generated code when using
PGO (Profile-Guided Optimization) or at optimization level 0.
-In Clang, the attributes will be ignored if they're not placed on the
-substatement of an ``if`` or ``else`` statement. The C++ Standard recommends
-to honor them on every statement in the path of execution, but that can be
-confusing:
+In Clang, the attributes will be ignored if they're not placed on
+* the ``case`` or ``default`` label of a ``switch`` statement,
+* or on the substatement of an ``if`` or ``else`` statement.
+The C++ Standard recommends to honor them on every statement in the
+path of execution, but that can be confusing:
.. code-block:: c++
@@ -1759,8 +1769,8 @@ confusing:
}
-At the moment the attribute only has effect when used in an ``if`` or ``else``
-statement.
+At the moment the attributes only have effect when used in an ``if``, ``else``,
+or ``switch`` statement.
.. code-block:: c++
@@ -1802,6 +1812,44 @@ statement.
[[likely]] int i = 5; // Issues a diagnostic since the attribute
// isn't allowed on a declaration.
+ switch (i) {
+ [[likely]] case 1: // This value is likely
+ ...
+ break;
+
+ [[unlikely]] case 2: // This value is unlikely
+ ...
+ [[fallthrough]];
+
+ case 3: // No likelihood attribute
+ ...
+ [[likely]] break; // No effect
+
+ case 4: [[likely]] { // attribute on substatement has no effect
+ ...
+ break;
+ }
+
+ [[unlikely]] default: // All other values are unlikely
+ ...
+ break;
+ }
+
+ switch (i) {
+ [[likely]] case 0: // This value and code path is likely
+ ...
+ [[fallthrough]];
+
+ case 1: // No likelihood attribute, code path is neutral
+ break; // falling through has no effect on the likelihood
+
+ case 2: // No likelihood attribute, code path is neutral
+ [[fallthrough]];
+
+ [[unlikely]] default: // This value and code path are both unlikely
+ break;
+ }
+
}];
}
diff --git a/clang/lib/AST/Stmt.cpp b/clang/lib/AST/Stmt.cpp
index e7024b2105fc..09503079ffae 100644
--- a/clang/lib/AST/Stmt.cpp
+++ b/clang/lib/AST/Stmt.cpp
@@ -130,19 +130,30 @@ void Stmt::EnableStatistics() {
StatisticsEnabled = true;
}
+static std::pair<Stmt::Likelihood, const Attr *>
+getLikelihood(ArrayRef<const Attr *> Attrs) {
+ for (const auto *A : Attrs) {
+ if (isa<LikelyAttr>(A))
+ return std::make_pair(Stmt::LH_Likely, A);
+
+ if (isa<UnlikelyAttr>(A))
+ return std::make_pair(Stmt::LH_Unlikely, A);
+ }
+
+ return std::make_pair(Stmt::LH_None, nullptr);
+}
+
static std::pair<Stmt::Likelihood, const Attr *> getLikelihood(const Stmt *S) {
if (const auto *AS = dyn_cast_or_null<AttributedStmt>(S))
- for (const auto *A : AS->getAttrs()) {
- if (isa<LikelyAttr>(A))
- return std::make_pair(Stmt::LH_Likely, A);
-
- if (isa<UnlikelyAttr>(A))
- return std::make_pair(Stmt::LH_Unlikely, A);
- }
+ return getLikelihood(AS->getAttrs());
return std::make_pair(Stmt::LH_None, nullptr);
}
+Stmt::Likelihood Stmt::getLikelihood(ArrayRef<const Attr *> Attrs) {
+ return ::getLikelihood(Attrs).first;
+}
+
Stmt::Likelihood Stmt::getLikelihood(const Stmt *S) {
return ::getLikelihood(S).first;
}
diff --git a/clang/lib/CodeGen/CGStmt.cpp b/clang/lib/CodeGen/CGStmt.cpp
index c9e6ce2df2c0..4b813be086e1 100644
--- a/clang/lib/CodeGen/CGStmt.cpp
+++ b/clang/lib/CodeGen/CGStmt.cpp
@@ -50,7 +50,7 @@ void CodeGenFunction::EmitStmt(const Stmt *S, ArrayRef<const Attr *> Attrs) {
PGO.setCurrentStmt(S);
// These statements have their own debug info handling.
- if (EmitSimpleStmt(S))
+ if (EmitSimpleStmt(S, Attrs))
return;
// Check if we are generating unreachable code.
@@ -370,23 +370,44 @@ void CodeGenFunction::EmitStmt(const Stmt *S, ArrayRef<const Attr *> Attrs) {
}
}
-bool CodeGenFunction::EmitSimpleStmt(const Stmt *S) {
+bool CodeGenFunction::EmitSimpleStmt(const Stmt *S,
+ ArrayRef<const Attr *> Attrs) {
switch (S->getStmtClass()) {
- default: return false;
- case Stmt::NullStmtClass: break;
- case Stmt::CompoundStmtClass: EmitCompoundStmt(cast<CompoundStmt>(*S)); break;
- case Stmt::DeclStmtClass: EmitDeclStmt(cast<DeclStmt>(*S)); break;
- case Stmt::LabelStmtClass: EmitLabelStmt(cast<LabelStmt>(*S)); break;
+ default:
+ return false;
+ case Stmt::NullStmtClass:
+ break;
+ case Stmt::CompoundStmtClass:
+ EmitCompoundStmt(cast<CompoundStmt>(*S));
+ break;
+ case Stmt::DeclStmtClass:
+ EmitDeclStmt(cast<DeclStmt>(*S));
+ break;
+ case Stmt::LabelStmtClass:
+ EmitLabelStmt(cast<LabelStmt>(*S));
+ break;
case Stmt::AttributedStmtClass:
- EmitAttributedStmt(cast<AttributedStmt>(*S)); break;
- case Stmt::GotoStmtClass: EmitGotoStmt(cast<GotoStmt>(*S)); break;
- case Stmt::BreakStmtClass: EmitBreakStmt(cast<BreakStmt>(*S)); break;
- case Stmt::ContinueStmtClass: EmitContinueStmt(cast<ContinueStmt>(*S)); break;
- case Stmt::DefaultStmtClass: EmitDefaultStmt(cast<DefaultStmt>(*S)); break;
- case Stmt::CaseStmtClass: EmitCaseStmt(cast<CaseStmt>(*S)); break;
- case Stmt::SEHLeaveStmtClass: EmitSEHLeaveStmt(cast<SEHLeaveStmt>(*S)); break;
+ EmitAttributedStmt(cast<AttributedStmt>(*S));
+ break;
+ case Stmt::GotoStmtClass:
+ EmitGotoStmt(cast<GotoStmt>(*S));
+ break;
+ case Stmt::BreakStmtClass:
+ EmitBreakStmt(cast<BreakStmt>(*S));
+ break;
+ case Stmt::ContinueStmtClass:
+ EmitContinueStmt(cast<ContinueStmt>(*S));
+ break;
+ case Stmt::DefaultStmtClass:
+ EmitDefaultStmt(cast<DefaultStmt>(*S), Attrs);
+ break;
+ case Stmt::CaseStmtClass:
+ EmitCaseStmt(cast<CaseStmt>(*S), Attrs);
+ break;
+ case Stmt::SEHLeaveStmtClass:
+ EmitSEHLeaveStmt(cast<SEHLeaveStmt>(*S));
+ break;
}
-
return true;
}
@@ -1221,7 +1242,8 @@ void CodeGenFunction::EmitContinueStmt(const ContinueStmt &S) {
/// EmitCaseStmtRange - If case statement range is not too big then
/// add multiple cases to switch instruction, one for each value within
/// the range. If range is too big then emit "if" condition check.
-void CodeGenFunction::EmitCaseStmtRange(const CaseStmt &S) {
+void CodeGenFunction::EmitCaseStmtRange(const CaseStmt &S,
+ ArrayRef<const Attr *> Attrs) {
assert(S.getRHS() && "Expected RHS value in CaseStmt");
llvm::APSInt LHS = S.getLHS()->EvaluateKnownConstInt(getContext());
@@ -1238,6 +1260,7 @@ void CodeGenFunction::EmitCaseStmtRange(const CaseStmt &S) {
if (LHS.isSigned() ? RHS.slt(LHS) : RHS.ult(LHS))
return;
+ Stmt::Likelihood LH = Stmt::getLikelihood(Attrs);
llvm::APInt Range = RHS - LHS;
// FIXME: parameters such as this should not be hardcoded.
if (Range.ult(llvm::APInt(Range.getBitWidth(), 64))) {
@@ -1252,6 +1275,9 @@ void CodeGenFunction::EmitCaseStmtRange(const CaseStmt &S) {
for (unsigned I = 0; I != NCases; ++I) {
if (SwitchWeights)
SwitchWeights->push_back(Weight + (Rem ? 1 : 0));
+ else if (SwitchLikelihood)
+ SwitchLikelihood->push_back(LH);
+
if (Rem)
Rem--;
SwitchInsn->addCase(Builder.getInt(LHS), CaseDest);
@@ -1289,7 +1315,9 @@ void CodeGenFunction::EmitCaseStmtRange(const CaseStmt &S) {
// need to update the weight for the default, ie, the first case, to include
// this case.
(*SwitchWeights)[0] += ThisCount;
- }
+ } else if (SwitchLikelihood)
+ Weights = createBranchWeights(LH);
+
Builder.CreateCondBr(Cond, CaseDest, FalseDest, Weights);
// Restore the appropriate insertion point.
@@ -1299,7 +1327,8 @@ void CodeGenFunction::EmitCaseStmtRange(const CaseStmt &S) {
Builder.ClearInsertionPoint();
}
-void CodeGenFunction::EmitCaseStmt(const CaseStmt &S) {
+void CodeGenFunction::EmitCaseStmt(const CaseStmt &S,
+ ArrayRef<const Attr *> Attrs) {
// If there is no enclosing switch instance that we're aware of, then this
// case statement and its block can be elided. This situation only happens
// when we've constant-folded the switch, are emitting the constant case,
@@ -1312,12 +1341,14 @@ void CodeGenFunction::EmitCaseStmt(const CaseStmt &S) {
// Handle case ranges.
if (S.getRHS()) {
- EmitCaseStmtRange(S);
+ EmitCaseStmtRange(S, Attrs);
return;
}
llvm::ConstantInt *CaseVal =
Builder.getInt(S.getLHS()->EvaluateKnownConstInt(getContext()));
+ if (SwitchLikelihood)
+ SwitchLikelihood->push_back(Stmt::getLikelihood(Attrs));
// If the body of the case is just a 'break', try to not emit an empty block.
// If we're profiling or we're not optimizing, leave the block in for better
@@ -1358,6 +1389,10 @@ void CodeGenFunction::EmitCaseStmt(const CaseStmt &S) {
// that falls through to the next case which is IR intensive. It also causes
// deep recursion which can run into stack depth limitations. Handle
// sequential non-range case statements specially.
+ //
+ // TODO When the next case has a likelihood attribute the code returns to the
+ // recursive algorithm. Maybe improve this case if it becomes common practice
+ // to use a lot of attributes.
const CaseStmt *CurCase = &S;
const CaseStmt *NextCase = dyn_cast<CaseStmt>(S.getSubStmt());
@@ -1373,6 +1408,10 @@ void CodeGenFunction::EmitCaseStmt(const CaseStmt &S) {
CaseDest = createBasicBlock("sw.bb");
EmitBlockWithFallThrough(CaseDest, &S);
}
+ // Since this loop is only executed when the CaseStmt has no attributes
+ // use a hard-coded value.
+ if (SwitchLikelihood)
+ SwitchLikelihood->push_back(Stmt::LH_None);
SwitchInsn->addCase(CaseVal, CaseDest);
NextCase = dyn_cast<CaseStmt>(CurCase->getSubStmt());
@@ -1382,7 +1421,8 @@ void CodeGenFunction::EmitCaseStmt(const CaseStmt &S) {
EmitStmt(CurCase->getSubStmt());
}
-void CodeGenFunction::EmitDefaultStmt(const DefaultStmt &S) {
+void CodeGenFunction::EmitDefaultStmt(const DefaultStmt &S,
+ ArrayRef<const Attr *> Attrs) {
// If there is no enclosing switch instance that we're aware of, then this
// default statement can be elided. This situation only happens when we've
// constant-folded the switch.
@@ -1395,6 +1435,9 @@ void CodeGenFunction::EmitDefaultStmt(const DefaultStmt &S) {
assert(DefaultBlock->empty() &&
"EmitDefaultStmt: Default block already defined?");
+ if (SwitchLikelihood)
+ SwitchLikelihood->front() = Stmt::getLikelihood(Attrs);
+
EmitBlockWithFallThrough(DefaultBlock, &S);
EmitStmt(S.getSubStmt());
@@ -1632,10 +1675,67 @@ static bool FindCaseStatementsForValue(const SwitchStmt &S,
FoundCase;
}
+static Optional<SmallVector<uint64_t, 16>>
+getLikelihoodWeights(ArrayRef<Stmt::Likelihood> Likelihoods) {
+ // Are there enough branches to weight them?
+ if (Likelihoods.size() <= 1)
+ return None;
+
+ uint64_t NumUnlikely = 0;
+ uint64_t NumNone = 0;
+ uint64_t NumLikely = 0;
+ for (const auto LH : Likelihoods) {
+ switch (LH) {
+ case Stmt::LH_Unlikely:
+ ++NumUnlikely;
+ break;
+ case Stmt::LH_None:
+ ++NumNone;
+ break;
+ case Stmt::LH_Likely:
+ ++NumLikely;
+ break;
+ }
+ }
+
+ // Is there a likelihood attribute used?
+ if (NumUnlikely == 0 && NumLikely == 0)
+ return None;
+
+ // When multiple cases share the same code they can be combined during
+ // optimization. In that case the weights of the branch will be the sum of
+ // the individual weights. Make sure the combined sum of all neutral cases
+ // doesn't exceed the value of a single likely attribute.
+ // The additions both avoid divisions by 0 and make sure the weights of None
+ // don't exceed the weight of Likely.
+ const uint64_t Likely = INT32_MAX / (NumLikely + 2);
+ const uint64_t None = Likely / (NumNone + 1);
+ const uint64_t Unlikely = 0;
+
+ SmallVector<uint64_t, 16> Result;
+ Result.reserve(Likelihoods.size());
+ for (const auto LH : Likelihoods) {
+ switch (LH) {
+ case Stmt::LH_Unlikely:
+ Result.push_back(Unlikely);
+ break;
+ case Stmt::LH_None:
+ Result.push_back(None);
+ break;
+ case Stmt::LH_Likely:
+ Result.push_back(Likely);
+ break;
+ }
+ }
+
+ return Result;
+}
+
void CodeGenFunction::EmitSwitchStmt(const SwitchStmt &S) {
// Handle nested switch statements.
llvm::SwitchInst *SavedSwitchInsn = SwitchInsn;
SmallVector<uint64_t, 16> *SavedSwitchWeights = SwitchWeights;
+ SmallVector<Stmt::Likelihood, 16> *SavedSwitchLikelihood = SwitchLikelihood;
llvm::BasicBlock *SavedCRBlock = CaseRangeBlock;
// See if we can constant fold the condition of the switch and therefore only
@@ -1710,7 +1810,12 @@ void CodeGenFunction::EmitSwitchStmt(const SwitchStmt &S) {
// The default needs to be first. We store the edge count, so we already
// know the right weight.
SwitchWeights->push_back(DefaultCount);
+ } else if (CGM.getCodeGenOpts().OptimizationLevel) {
+ SwitchLikelihood = new SmallVector<Stmt::Likelihood, 16>();
+ // Initialize the default case.
+ SwitchLikelihood->push_back(Stmt::LH_None);
}
+
CaseRangeBlock = DefaultBlock;
// Clear the insertion point to indicate we are in unreachable code.
@@ -1774,9 +1879,21 @@ void CodeGenFunction::EmitSwitchStmt(const SwitchStmt &S) {
SwitchInsn->setMetadata(llvm::LLVMContext::MD_prof,
createProfileWeights(*SwitchWeights));
delete SwitchWeights;
+ } else if (SwitchLikelihood) {
+ assert(SwitchLikelihood->size() == 1 + SwitchInsn->getNumCases() &&
+ "switch likelihoods do not match switch cases");
+ Optional<SmallVector<uint64_t, 16>> LHW =
+ getLikelihoodWeights(*SwitchLikelihood);
+ if (LHW) {
+ llvm::MDBuilder MDHelper(CGM.getLLVMContext());
+ SwitchInsn->setMetadata(llvm::LLVMContext::MD_prof,
+ createProfileWeights(*LHW));
+ }
+ delete SwitchLikelihood;
}
SwitchInsn = SavedSwitchInsn;
SwitchWeights = SavedSwitchWeights;
+ SwitchLikelihood = SavedSwitchLikelihood;
CaseRangeBlock = SavedCRBlock;
}
diff --git a/clang/lib/CodeGen/CodeGenFunction.cpp b/clang/lib/CodeGen/CodeGenFunction.cpp
index 363b418dc198..2dcb5314c61e 100644
--- a/clang/lib/CodeGen/CodeGenFunction.cpp
+++ b/clang/lib/CodeGen/CodeGenFunction.cpp
@@ -1478,21 +1478,6 @@ bool CodeGenFunction::ConstantFoldsToSimpleInteger(const Expr *Cond,
return true;
}
-static Optional<std::pair<uint32_t, uint32_t>>
-getLikelihoodWeights(Stmt::Likelihood LH) {
- switch (LH) {
- case Stmt::LH_Unlikely:
- return std::pair<uint32_t, uint32_t>(llvm::UnlikelyBranchWeight,
- llvm::LikelyBranchWeight);
- case Stmt::LH_None:
- return None;
- case Stmt::LH_Likely:
- return std::pair<uint32_t, uint32_t>(llvm::LikelyBranchWeight,
- llvm::UnlikelyBranchWeight);
- }
- llvm_unreachable("Unknown Likelihood");
-}
-
/// EmitBranchOnBoolExpr - Emit a branch on a boolean condition (e.g. for an if
/// statement) to the specified blocks. Based on the condition, this might try
/// to simplify the codegen of the conditional based on the branch.
@@ -1692,12 +1677,7 @@ void CodeGenFunction::EmitBranchOnBoolExpr(const Expr *Cond,
}
}
- llvm::MDNode *Weights = nullptr;
- Optional<std::pair<uint32_t, uint32_t>> LHW = getLikelihoodWeights(LH);
- if (LHW) {
- llvm::MDBuilder MDHelper(CGM.getLLVMContext());
- Weights = MDHelper.createBranchWeights(LHW->first, LHW->second);
- }
+ llvm::MDNode *Weights = createBranchWeights(LH);
if (!Weights) {
uint64_t CurrentCount = std::max(getCurrentProfileCount(), TrueCount);
Weights = createProfileWeights(TrueCount, CurrentCount - TrueCount);
@@ -2569,3 +2549,27 @@ llvm::DebugLoc CodeGenFunction::SourceLocToDebugLoc(SourceLocation Location) {
return llvm::DebugLoc();
}
+
+static Optional<std::pair<uint32_t, uint32_t>>
+getLikelihoodWeights(Stmt::Likelihood LH) {
+ switch (LH) {
+ case Stmt::LH_Unlikely:
+ return std::pair<uint32_t, uint32_t>(llvm::UnlikelyBranchWeight,
+ llvm::LikelyBranchWeight);
+ case Stmt::LH_None:
+ return None;
+ case Stmt::LH_Likely:
+ return std::pair<uint32_t, uint32_t>(llvm::LikelyBranchWeight,
+ llvm::UnlikelyBranchWeight);
+ }
+ llvm_unreachable("Unknown Likelihood");
+}
+
+llvm::MDNode *CodeGenFunction::createBranchWeights(Stmt::Likelihood LH) const {
+ Optional<std::pair<uint32_t, uint32_t>> LHW = getLikelihoodWeights(LH);
+ if (!LHW)
+ return nullptr;
+
+ llvm::MDBuilder MDHelper(CGM.getLLVMContext());
+ return MDHelper.createBranchWeights(LHW->first, LHW->second);
+}
diff --git a/clang/lib/CodeGen/CodeGenFunction.h b/clang/lib/CodeGen/CodeGenFunction.h
index a9fb47864700..0a199e9b86ec 100644
--- a/clang/lib/CodeGen/CodeGenFunction.h
+++ b/clang/lib/CodeGen/CodeGenFunction.h
@@ -1395,6 +1395,9 @@ class CodeGenFunction : public CodeGenTypeCache {
};
OpenMPCancelExitStack OMPCancelStack;
+ /// Calculate branch weights for the likelihood attribute
+ llvm::MDNode *createBranchWeights(Stmt::Likelihood LH) const;
+
CodeGenPGO PGO;
/// Calculate branch weights appropriate for PGO data
@@ -1439,6 +1442,9 @@ class CodeGenFunction : public CodeGenTypeCache {
/// The branch weights of SwitchInsn when doing instrumentation based PGO.
SmallVector<uint64_t, 16> *SwitchWeights = nullptr;
+ /// The likelihood attributes of the SwitchCase.
+ SmallVector<Stmt::Likelihood, 16> *SwitchLikelihood = nullptr;
+
/// CaseRangeBlock - This block holds if condition check for last case
/// statement range in current switch instruction.
llvm::BasicBlock *CaseRangeBlock = nullptr;
@@ -3075,7 +3081,7 @@ class CodeGenFunction : public CodeGenTypeCache {
/// statements.
///
/// \return True if the statement was handled.
- bool EmitSimpleStmt(const Stmt *S);
+ bool EmitSimpleStmt(const Stmt *S, ArrayRef<const Attr *> Attrs);
Address EmitCompoundStmt(const CompoundStmt &S, bool GetLast = false,
AggValueSlot AVS = AggValueSlot::ignored());
@@ -3104,9 +3110,9 @@ class CodeGenFunction : public CodeGenTypeCache {
void EmitBreakStmt(const BreakStmt &S);
void EmitContinueStmt(const ContinueStmt &S);
void EmitSwitchStmt(const SwitchStmt &S);
- void EmitDefaultStmt(const DefaultStmt &S);
- void EmitCaseStmt(const CaseStmt &S);
- void EmitCaseStmtRange(const CaseStmt &S);
+ void EmitDefaultStmt(const DefaultStmt &S, ArrayRef<const Attr *> Attrs);
+ void EmitCaseStmt(const CaseStmt &S, ArrayRef<const Attr *> Attrs);
+ void EmitCaseStmtRange(const CaseStmt &S, ArrayRef<const Attr *> Attrs);
void EmitAsmStmt(const AsmStmt &S);
void EmitObjCForCollectionStmt(const ObjCForCollectionStmt &S);
diff --git a/clang/test/CodeGenCXX/attr-likelihood-switch-branch-weights.cpp b/clang/test/CodeGenCXX/attr-likelihood-switch-branch-weights.cpp
new file mode 100644
index 000000000000..5fb7a67a7d9e
--- /dev/null
+++ b/clang/test/CodeGenCXX/attr-likelihood-switch-branch-weights.cpp
@@ -0,0 +1,194 @@
+// RUN: %clang_cc1 -O1 -disable-llvm-passes -emit-llvm %s -o - -triple=x86_64-linux-gnu | FileCheck %s
+
+extern volatile int i;
+
+void OneCaseL() {
+ // CHECK-LABEL: define{{.*}}OneCaseL
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !6
+ switch (i) {
+ [[likely]] case 1: break;
+ }
+}
+
+void OneCaseU() {
+ // CHECK-LABEL: define{{.*}}OneCaseU
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !7
+ switch (i) {
+ [[unlikely]] case 1: ++i; break;
+ }
+}
+
+void TwoCasesLN() {
+ // CHECK-LABEL: define{{.*}}TwoCasesLN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !8
+ switch (i) {
+ [[likely]] case 1: break;
+ case 2: break;
+ }
+}
+
+void TwoCasesUN() {
+ // CHECK-LABEL: define{{.*}}TwoCasesUN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !9
+ switch (i) {
+ [[unlikely]] case 1: break;
+ case 2: break;
+ }
+}
+
+void TwoCasesLU() {
+ // CHECK-LABEL: define{{.*}}TwoCasesLU
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !10
+ switch (i) {
+ [[likely]] case 1: break;
+ [[unlikely]] case 2: break;
+ }
+}
+
+void CasesFallthroughNNLN() {
+ // CHECK-LABEL: define{{.*}}CasesFallthroughNNLN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !11
+ switch (i) {
+ case 1:
+ case 2:
+ [[likely]] case 3:
+ case 4: break;
+ }
+}
+
+void CasesFallthroughNNUN() {
+ // CHECK-LABEL: define{{.*}}CasesFallthroughNNUN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !12
+ switch (i) {
+ case 1:
+ case 2:
+ [[unlikely]] case 3:
+ case 4: break;
+ }
+}
+
+void CasesFallthroughRangeSmallLN() {
+ // CHECK-LABEL: define{{.*}}CasesFallthroughRangeSmallLN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !13
+ switch (i) {
+ case 1 ... 5: ++i;
+ case 102:
+ [[likely]] case 103:
+ case 104: break;
+ }
+}
+
+void CasesFallthroughRangeSmallUN() {
+ // CHECK-LABEL: define{{.*}}CasesFallthroughRangeSmallUN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !14
+ switch (i) {
+ case 1 ... 5: ++i;
+ case 102:
+ [[unlikely]] case 103:
+ case 104: break;
+ }
+}
+
+void CasesFallthroughRangeLargeLLN() {
+ // CHECK-LABEL: define{{.*}}CasesFallthroughRangeLargeLLN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !8
+ // CHECK: caserange
+ // CHECK: br{{.*}} !prof !15
+ switch (i) {
+ [[likely]] case 0 ... 64:
+ [[likely]] case 1003:
+ case 104: break;
+ }
+}
+
+void CasesFallthroughRangeLargeUUN() {
+ // CHECK-LABEL: define{{.*}}CasesFallthroughRangeLargeUUN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !9
+ // CHECK: caserange
+ // CHECK: br{{.*}} !prof !16
+ switch (i) {
+ [[unlikely]] case 0 ... 64:
+ [[unlikely]] case 1003:
+ case 104: break;
+ }
+}
+
+void OneCaseDefaultL() {
+ // CHECK-LABEL: define{{.*}}OneCaseDefaultL
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !17
+ switch (i) {
+ case 1: break;
+ [[likely]] default: break;
+ }
+}
+
+void OneCaseDefaultU() {
+ // CHECK-LABEL: define{{.*}}OneCaseDefaultU
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !18
+ switch (i) {
+ case 1: break;
+ [[unlikely]] default: break;
+ }
+}
+
+void TwoCasesDefaultLNL() {
+ // CHECK-LABEL: define{{.*}}TwoCasesDefaultLNL
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !19
+ switch (i) {
+ [[likely]] case 1: break;
+ case 2: break;
+ [[likely]] default: break;
+ }
+}
+
+void TwoCasesDefaultLNN() {
+ // CHECK-LABEL: define{{.*}}TwoCasesDefaultLNN
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !8
+ switch (i) {
+ [[likely]] case 1: break;
+ case 2: break;
+ default: break;
+ }
+}
+
+void TwoCasesDefaultLNU() {
+ // CHECK-LABEL: define{{.*}}TwoCasesDefaultLNU
+ // CHECK: switch
+ // CHECK: {{.*}} !prof !20
+ switch (i) {
+ [[likely]] case 1: break;
+ case 2: break;
+ [[unlikely]] default: break;
+ }
+}
+
+// CHECK: !6 = !{!"branch_weights", i32 357913942, i32 715827883}
+// CHECK: !7 = !{!"branch_weights", i32 536870912, i32 1}
+// CHECK: !8 = !{!"branch_weights", i32 238609295, i32 715827883, i32 238609295}
+// CHECK: !9 = !{!"branch_weights", i32 357913942, i32 1, i32 357913942}
+// CHECK: !10 = !{!"branch_weights", i32 357913942, i32 715827883, i32 1}
+// CHECK: !11 = !{!"branch_weights", i32 143165577, i32 143165577, i32 143165577, i32 715827883, i32 143165577}
+// CHECK: !12 = !{!"branch_weights", i32 214748365, i32 214748365, i32 214748365, i32 1, i32 214748365}
+// CHECK: !13 = !{!"branch_weights", i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 79536432, i32 715827883, i32 79536432}
+// CHECK: !14 = !{!"branch_weights", i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 119304648, i32 1, i32 119304648}
+// CHECK: !15 = !{!"branch_weights", i32 2000, i32 1}
+// CHECK: !16 = !{!"branch_weights", i32 1, i32 2000}
+// CHECK: !17 = !{!"branch_weights", i32 715827883, i32 357913942}
+// CHECK: !18 = !{!"branch_weights", i32 1, i32 536870912}
+// CHECK: !19 = !{!"branch_weights", i32 536870912, i32 536870912, i32 268435456}
+// CHECK: !20 = !{!"branch_weights", i32 1, i32 715827883, i32 357913942}
diff --git a/clang/test/Preprocessor/has_attribute.cpp b/clang/test/Preprocessor/has_attribute.cpp
index a66624ac4147..fe7d29f15de1 100644
--- a/clang/test/Preprocessor/has_attribute.cpp
+++ b/clang/test/Preprocessor/has_attribute.cpp
@@ -62,13 +62,13 @@ CXX11(unlikely)
// FIXME(201806L) CHECK: ensures: 0
// FIXME(201806L) CHECK: expects: 0
// CHECK: fallthrough: 201603L
-// FIXME(201803L) CHECK: likely: 2L
+// CHECK: likely: 201803L
// CHECK: maybe_unused: 201603L
// ITANIUM: no_unique_address: 201803L
// WINDOWS: no_unique_address: 0
// CHECK: nodiscard: 201907L
// CHECK: noreturn: 200809L
-// FIXME(201803L) CHECK: unlikely: 2L
+// CHECK: unlikely: 201803L
// Test for Microsoft __declspec attributes
diff --git a/clang/www/cxx_status.html b/clang/www/cxx_status.html
index 9c39e396edd4..de6c5eff7609 100755
--- a/clang/www/cxx_status.html
+++ b/clang/www/cxx_status.html
@@ -988,7 +988,7 @@ <h2 id="cxx20">C++20 implementation status</h2>
<tr>
<td><tt>[[likely]]</tt> and <tt>[[unlikely]]</tt> attributes</td>
<td><a href="https://wg21.link/p0479r5">P0479R5</a></td>
- <td class="none" align="center">Clang 12 (partial)</td>
+ <td class="unreleased" align="center">Clang 12</td>
</tr>
<tr>
<td><tt>typename</tt> optional in more contexts</td>
More information about the cfe-commits
mailing list