[llvm-branch-commits] [clang] 5b6216d - [RISCV] Lazily add RVV C intrinsics.
Kito Cheng via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Wed May 11 02:57:53 PDT 2022
Author: Kito Cheng
Date: 2022-05-11T17:56:59+08:00
New Revision: 5b6216d6aa45c91bd348393eba8952f34735b736
URL: https://github.com/llvm/llvm-project/commit/5b6216d6aa45c91bd348393eba8952f34735b736
DIFF: https://github.com/llvm/llvm-project/commit/5b6216d6aa45c91bd348393eba8952f34735b736.diff
LOG: [RISCV] Lazily add RVV C intrinsics.
Leverage the method OpenCL uses that adds C intrinsics when the lookup
failed. There is no need to define C intrinsics in the header file any
more. It could help to avoid the large header file to speed up the
compilation of RVV source code. Besides that, only the C intrinsics used
by the users will be added into the declaration table.
This patch is based on https://reviews.llvm.org/D103228 and inspired by
OpenCL implementation.
### Experimental Results
#### TL;DR:
- Binary size of clang increase ~200k, which is +0.07% for debug build and +0.13% for release build.
- Single file compilation speed up ~33x for debug build and ~8.5x for release build
- Regression time reduce ~10% (`ninja check-all`, enable all targets)
#### Header size change
```
| size | LoC |
------------------------------
Before | 4,434,725 | 69,749 |
After | 6,140 | 162 |
```
#### Single File Compilation Time
Testcase:
```
#include <riscv_vector.h>
vint32m1_t test_vadd_vv_vfloat32m1_t(vint32m1_t op1, vint32m1_t op2, size_t vl) {
return vadd(op1, op2, vl);
}
```
##### Debug build:
Before:
```
real 0m19.352s
user 0m19.252s
sys 0m0.092s
```
After:
```
real 0m0.576s
user 0m0.552s
sys 0m0.024s
```
~33x speed up for debug build
##### Release build:
Before:
```
real 0m0.773s
user 0m0.741s
sys 0m0.032s
```
After:
```
real 0m0.092s
user 0m0.080s
sys 0m0.012s
```
~8.5x speed up for release build
#### Regression time
Note: the failed case is `tools/llvm-debuginfod-find/debuginfod.test` which is unrelated to this patch.
##### Debug build
Before:
```
Testing Time: 1358.38s
Skipped : 11
Unsupported : 446
Passed : 75767
Expectedly Failed: 190
Failed : 1
```
After
```
Testing Time: 1220.29s
Skipped : 11
Unsupported : 446
Passed : 75767
Expectedly Failed: 190
Failed : 1
```
##### Release build
Before:
```
Testing Time: 381.98s
Skipped : 12
Unsupported : 1407
Passed : 74765
Expectedly Failed: 176
Failed : 1
```
After:
```
Testing Time: 346.25s
Skipped : 12
Unsupported : 1407
Passed : 74765
Expectedly Failed: 176
Failed : 1
```
#### Binary size of clang
##### Debug build
Before
```
text data bss dec hex filename
335261851 12726004 552812 348540667 14c64efb bin/clang
```
After
```
text data bss dec hex filename
335442803 12798708 552940 348794451 14ca2e53 bin/clang
```
+253K, +0.07% code size
##### Release build
Before
```
text data bss dec hex filename
144123975 8374648 483140 152981763 91e5103 bin/clang
```
After
```
text data bss dec hex filename
144255762 8447296 483268 153186326 9217016 bin/clang
```
+204K, +0.13%
Authored-by: Kito Cheng <kito.cheng at sifive.com>
Co-Authored-by: Hsiangkai Wang <kai.wang at sifive.com>
Differential Revision: https://reviews.llvm.org/D111617
Added:
clang/lib/Sema/SemaRVVLookup.cpp
Modified:
clang/include/clang/Basic/CMakeLists.txt
clang/include/clang/Basic/TokenKinds.def
clang/include/clang/Parse/Parser.h
clang/include/clang/Sema/Sema.h
clang/include/clang/Support/RISCVVIntrinsicUtils.h
clang/lib/Parse/ParsePragma.cpp
clang/lib/Sema/CMakeLists.txt
clang/lib/Sema/SemaLookup.cpp
clang/lib/Support/RISCVVIntrinsicUtils.cpp
clang/utils/TableGen/RISCVVEmitter.cpp
clang/utils/TableGen/TableGen.cpp
clang/utils/TableGen/TableGenBackends.h
Removed:
################################################################################
diff --git a/clang/include/clang/Basic/CMakeLists.txt b/clang/include/clang/Basic/CMakeLists.txt
index 8cd891385a483..b930842ae8cfd 100644
--- a/clang/include/clang/Basic/CMakeLists.txt
+++ b/clang/include/clang/Basic/CMakeLists.txt
@@ -90,3 +90,6 @@ clang_tablegen(riscv_vector_builtins.inc -gen-riscv-vector-builtins
clang_tablegen(riscv_vector_builtin_cg.inc -gen-riscv-vector-builtin-codegen
SOURCE riscv_vector.td
TARGET ClangRISCVVectorBuiltinCG)
+clang_tablegen(riscv_vector_builtin_sema.inc -gen-riscv-vector-builtin-sema
+ SOURCE riscv_vector.td
+ TARGET ClangRISCVVectorBuiltinSema)
diff --git a/clang/include/clang/Basic/TokenKinds.def b/clang/include/clang/Basic/TokenKinds.def
index 7b65a15378050..fc159ab3114d0 100644
--- a/clang/include/clang/Basic/TokenKinds.def
+++ b/clang/include/clang/Basic/TokenKinds.def
@@ -908,6 +908,9 @@ PRAGMA_ANNOTATION(pragma_fp)
// Annotation for the attribute pragma directives - #pragma clang attribute ...
PRAGMA_ANNOTATION(pragma_attribute)
+// Annotation for the riscv pragma directives - #pragma clang riscv intrinsic ..
+PRAGMA_ANNOTATION(pragma_riscv)
+
// Annotations for module import translated from #include etc.
ANNOTATION(module_include)
ANNOTATION(module_begin)
diff --git a/clang/include/clang/Parse/Parser.h b/clang/include/clang/Parse/Parser.h
index caa58d9260628..c4e9716c5d9c7 100644
--- a/clang/include/clang/Parse/Parser.h
+++ b/clang/include/clang/Parse/Parser.h
@@ -213,6 +213,7 @@ class Parser : public CodeCompletionHandler {
std::unique_ptr<PragmaHandler> AttributePragmaHandler;
std::unique_ptr<PragmaHandler> MaxTokensHerePragmaHandler;
std::unique_ptr<PragmaHandler> MaxTokensTotalPragmaHandler;
+ std::unique_ptr<PragmaHandler> RISCVPragmaHandler;
std::unique_ptr<CommentHandler> CommentSemaHandler;
diff --git a/clang/include/clang/Sema/Sema.h b/clang/include/clang/Sema/Sema.h
index 27603f0b891f3..e2643f36c39dd 100644
--- a/clang/include/clang/Sema/Sema.h
+++ b/clang/include/clang/Sema/Sema.h
@@ -1576,6 +1576,9 @@ class Sema final {
/// assignment.
llvm::DenseMap<const VarDecl *, int> RefsMinusAssignments;
+ /// Indicate RVV builtin funtions enabled or not.
+ bool DeclareRVVBuiltins = false;
+
private:
Optional<std::unique_ptr<DarwinSDKInfo>> CachedDarwinSDKInfo;
@@ -13370,6 +13373,8 @@ void Sema::PragmaStack<Sema::AlignPackInfo>::Act(SourceLocation PragmaLocation,
llvm::StringRef StackSlotLabel,
AlignPackInfo Value);
+bool GetRVVBuiltinInfo(Sema &S, LookupResult &LR, IdentifierInfo *II,
+ Preprocessor &PP);
} // end namespace clang
namespace llvm {
diff --git a/clang/include/clang/Support/RISCVVIntrinsicUtils.h b/clang/include/clang/Support/RISCVVIntrinsicUtils.h
index ddd46fe1727c9..63a99b02bd572 100644
--- a/clang/include/clang/Support/RISCVVIntrinsicUtils.h
+++ b/clang/include/clang/Support/RISCVVIntrinsicUtils.h
@@ -218,6 +218,12 @@ class RVVType {
bool isFloat(unsigned Width) const {
return isFloat() && ElementBitwidth == Width;
}
+ bool isConstant() const { return IsConstant; }
+ bool isPointer() const { return IsPointer; }
+ unsigned getElementBitwidth() const { return ElementBitwidth; }
+
+ ScalarTypeKind getScalarType() const { return ScalarType; }
+ VScaleVal getScale() const { return Scale; }
private:
// Verify RVV vector type and set Valid.
@@ -255,18 +261,6 @@ class RVVType {
TypeProfile Proto);
};
-using RISCVPredefinedMacroT = uint8_t;
-
-enum RISCVPredefinedMacro : RISCVPredefinedMacroT {
- Basic = 0,
- V = 1 << 1,
- Zvfh = 1 << 2,
- RV64 = 1 << 3,
- VectorMaxELen64 = 1 << 4,
- VectorMaxELenFp32 = 1 << 5,
- VectorMaxELenFp64 = 1 << 6,
-};
-
enum PolicyScheme : uint8_t {
SchemeNone,
HasPassthruOperand,
@@ -294,7 +288,6 @@ class RVVIntrinsic {
// The types we use to obtain the specific LLVM intrinsic. They are index of
// InputTypes. -1 means the return type.
std::vector<int64_t> IntrinsicTypes;
- RISCVPredefinedMacroT RISCVPredefinedMacros = 0;
unsigned NF = 1;
public:
@@ -323,9 +316,6 @@ class RVVIntrinsic {
llvm::StringRef getIRName() const { return IRName; }
llvm::StringRef getManualCodegen() const { return ManualCodegen; }
PolicyScheme getPolicyScheme() const { return Scheme; }
- RISCVPredefinedMacroT getRISCVPredefinedMacros() const {
- return RISCVPredefinedMacros;
- }
unsigned getNF() const { return NF; }
const std::vector<int64_t> &getIntrinsicTypes() const {
return IntrinsicTypes;
@@ -336,7 +326,7 @@ class RVVIntrinsic {
static std::string
getSuffixStr(BasicType Type, int Log2LMUL,
- const llvm::SmallVector<TypeProfile> &TypeProfiles);
+ const llvm::ArrayRef<TypeProfile> &TypeProfiles);
};
} // end namespace RISCV
diff --git a/clang/lib/Parse/ParsePragma.cpp b/clang/lib/Parse/ParsePragma.cpp
index 75577f8f4e04c..9f9d55618f150 100644
--- a/clang/lib/Parse/ParsePragma.cpp
+++ b/clang/lib/Parse/ParsePragma.cpp
@@ -356,6 +356,16 @@ struct PragmaMaxTokensTotalHandler : public PragmaHandler {
Token &FirstToken) override;
};
+struct PragmaRISCVHandler : public PragmaHandler {
+ PragmaRISCVHandler(Sema &Actions)
+ : PragmaHandler("riscv"), Actions(Actions) {}
+ void HandlePragma(Preprocessor &PP, PragmaIntroducer Introducer,
+ Token &FirstToken) override;
+
+private:
+ Sema &Actions;
+};
+
void markAsReinjectedForRelexing(llvm::MutableArrayRef<clang::Token> Toks) {
for (auto &T : Toks)
T.setFlag(clang::Token::IsReinjected);
@@ -495,6 +505,11 @@ void Parser::initializePragmaHandlers() {
MaxTokensTotalPragmaHandler = std::make_unique<PragmaMaxTokensTotalHandler>();
PP.AddPragmaHandler("clang", MaxTokensTotalPragmaHandler.get());
+
+ if (getTargetInfo().getTriple().isRISCV()) {
+ RISCVPragmaHandler = std::make_unique<PragmaRISCVHandler>(Actions);
+ PP.AddPragmaHandler("clang", RISCVPragmaHandler.get());
+ }
}
void Parser::resetPragmaHandlers() {
@@ -615,6 +630,11 @@ void Parser::resetPragmaHandlers() {
PP.RemovePragmaHandler("clang", MaxTokensTotalPragmaHandler.get());
MaxTokensTotalPragmaHandler.reset();
+
+ if (getTargetInfo().getTriple().isRISCV()) {
+ PP.RemovePragmaHandler("clang", RISCVPragmaHandler.get());
+ RISCVPragmaHandler.reset();
+ }
}
/// Handle the annotation token produced for #pragma unused(...)
@@ -3830,3 +3850,34 @@ void PragmaMaxTokensTotalHandler::HandlePragma(Preprocessor &PP,
PP.overrideMaxTokens(MaxTokens, Loc);
}
+
+// Handle '#pragma clang riscv intrinsic vector'.
+void PragmaRISCVHandler::HandlePragma(Preprocessor &PP,
+ PragmaIntroducer Introducer,
+ Token &FirstToken) {
+ Token Tok;
+ PP.Lex(Tok);
+ IdentifierInfo *II = Tok.getIdentifierInfo();
+ if (!II || (!II->isStr("intrinsic"))) {
+ PP.Diag(Tok.getLocation(), diag::warn_pragma_invalid_argument)
+ << PP.getSpelling(Tok) << "riscv" << /*Expected=*/true << "'intrinsic'";
+ return;
+ }
+
+ PP.Lex(Tok);
+ II = Tok.getIdentifierInfo();
+ if (!II || (!II->isStr("vector"))) {
+ PP.Diag(Tok.getLocation(), diag::warn_pragma_invalid_argument)
+ << PP.getSpelling(Tok) << "riscv" << /*Expected=*/true << "'vector'";
+ return;
+ }
+
+ PP.Lex(Tok);
+ if (Tok.isNot(tok::eod)) {
+ PP.Diag(Tok.getLocation(), diag::warn_pragma_extra_tokens_at_eol)
+ << "clang riscv intrinsic";
+ return;
+ }
+
+ Actions.DeclareRVVBuiltins = true;
+}
diff --git a/clang/lib/Sema/CMakeLists.txt b/clang/lib/Sema/CMakeLists.txt
index 0e0681a8e2927..9a37fbbe59fe5 100644
--- a/clang/lib/Sema/CMakeLists.txt
+++ b/clang/lib/Sema/CMakeLists.txt
@@ -46,6 +46,7 @@ add_clang_library(clangSema
SemaInit.cpp
SemaLambda.cpp
SemaLookup.cpp
+ SemaRVVLookup.cpp
SemaModule.cpp
SemaObjCProperty.cpp
SemaOpenMP.cpp
@@ -73,4 +74,5 @@ add_clang_library(clangSema
clangBasic
clangEdit
clangLex
+ clangSupport
)
diff --git a/clang/lib/Sema/SemaLookup.cpp b/clang/lib/Sema/SemaLookup.cpp
index 9f2e1eac41be6..48b6d0eccfb58 100644
--- a/clang/lib/Sema/SemaLookup.cpp
+++ b/clang/lib/Sema/SemaLookup.cpp
@@ -928,6 +928,12 @@ bool Sema::LookupBuiltin(LookupResult &R) {
}
}
+ if (DeclareRVVBuiltins) {
+ if (GetRVVBuiltinInfo(*this, R, II, PP)) {
+ return true;
+ }
+ }
+
// If this is a builtin on this (or all) targets, create the decl.
if (unsigned BuiltinID = II->getBuiltinID()) {
// In C++, C2x, and OpenCL (spec v1.2 s6.9.f), we don't have any
diff --git a/clang/lib/Sema/SemaRVVLookup.cpp b/clang/lib/Sema/SemaRVVLookup.cpp
new file mode 100644
index 0000000000000..7db3913afbbe2
--- /dev/null
+++ b/clang/lib/Sema/SemaRVVLookup.cpp
@@ -0,0 +1,469 @@
+//===-- SemaRVVLookup.cpp - Name Lookup for RISC-V Vector Intrinsic -------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements name lookup for RISC-V vector intrinsic.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/AST/ASTContext.h"
+#include "clang/AST/CXXInheritance.h"
+#include "clang/AST/Decl.h"
+#include "clang/AST/DeclCXX.h"
+#include "clang/AST/DeclLookups.h"
+#include "clang/AST/DeclObjC.h"
+#include "clang/AST/DeclTemplate.h"
+#include "clang/AST/Expr.h"
+#include "clang/AST/ExprCXX.h"
+#include "clang/Basic/Builtins.h"
+#include "clang/Basic/FileManager.h"
+#include "clang/Basic/LangOptions.h"
+#include "clang/Basic/TargetBuiltins.h"
+#include "clang/Basic/TargetInfo.h"
+#include "clang/Lex/HeaderSearch.h"
+#include "clang/Lex/ModuleLoader.h"
+#include "clang/Lex/Preprocessor.h"
+#include "clang/Sema/DeclSpec.h"
+#include "clang/Sema/Lookup.h"
+#include "clang/Sema/Overload.h"
+#include "clang/Sema/Scope.h"
+#include "clang/Sema/ScopeInfo.h"
+#include "clang/Sema/Sema.h"
+#include "clang/Sema/SemaInternal.h"
+#include "clang/Sema/TemplateDeduction.h"
+#include "clang/Sema/TypoCorrection.h"
+#include "clang/Support/RISCVVIntrinsicUtils.h"
+#include "llvm/ADT/BitmaskEnum.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/ADT/TinyPtrVector.h"
+#include "llvm/ADT/edit_distance.h"
+#include "llvm/Support/ErrorHandling.h"
+#include <algorithm>
+#include <iterator>
+#include <list>
+#include <set>
+#include <utility>
+#include <vector>
+
+using namespace llvm;
+using namespace clang;
+using namespace clang::RISCV;
+
+namespace {
+
+// RVVRequire should be sync with target features, but only
+// required features used in riscv_vector.td.
+enum class RVVRequire : uint8_t {
+ None = 0,
+ RV64 = 1 << 0,
+ FullMultiply = 1 << 1,
+
+ LLVM_MARK_AS_BITMASK_ENUM(FullMultiply)
+};
+
+// Raw RVV intrinsic info, used to expand later.
+// This struct is highly compact for minimized code size, and should sync
+// with RISCVVEmitter.cpp.
+struct RVVIntrinsicRecord {
+ // Intrinsic name, e.g. vadd_vv
+ const char *Name;
+
+ // Overloaded intrinsic name, could be empty if can be computed from Name
+ // e.g. vadd
+ const char *OverloadedName;
+
+ // Prototype for this intrinsic, index of RVVSignatureTable.
+ uint16_t PrototypeIndex;
+
+ // Prototype for masked intrinsic, index of RVVSignatureTable.
+ uint16_t MaskedPrototypeIndex;
+
+ // Suffix of intrinsic name, index of RVVSignatureTable.
+ uint16_t SuffixIndex;
+
+ // Suffix of overloaded intrinsic name, index of RVVSignatureTable.
+ uint16_t OverloadedSuffixIndex;
+
+ // Length of the prototype.
+ uint8_t PrototypeLength;
+
+ // Length of prototype of masked intrinsic.
+ uint8_t MaskedPrototypeLength;
+
+ // Length of intrinsic name suffix.
+ uint8_t SuffixLength;
+
+ // Length of overloaded intrinsic suffix.
+ uint8_t OverloadedSuffixSize;
+
+ // Required target features for this intrinsic.
+ RVVRequire RequiredExtension;
+
+ // Supported type, mask of BasicType
+ uint8_t TypeRangeMask;
+
+ // Supported LMUL.
+ uint8_t Log2LMULMask;
+
+ // Number of field, large than 1 if it's segment load/store.
+ uint8_t NF;
+};
+
+// Function definition of a RVV intrinsic
+struct RVVIntrinsicDef {
+ /// Full function name with suffix, e.g. vadd_vv_i32m1.
+ std::string Name;
+
+ /// Overloaded function name, e.g. vadd.
+ std::string OverloadName;
+
+ /// Mapping to which clang built-in function, e.g. __builtin_rvv_vadd
+ std::string BuiltinName;
+
+ /// Function signature, first element is return type.
+ RVVTypes Signature;
+};
+
+struct RVVOverloadIntrinsicDef {
+ // Index of RVVIntrinsicManager::IntrinsicList.
+ SmallVector<size_t, 8> Indexs;
+};
+
+LLVM_ENABLE_BITMASK_ENUMS_IN_NAMESPACE();
+} // namespace
+
+static const TypeProfile RVVSignatureTable[] = {
+#define DECL_SIGNATURE_TABLE
+#include "clang/Basic/riscv_vector_builtin_sema.inc"
+#undef DECL_SIGNATURE_TABLE
+};
+
+static const RVVIntrinsicRecord RVVIntrinsicRecords[] = {
+#define DECL_INTRINSIC_RECORDS
+#include "clang/Basic/riscv_vector_builtin_sema.inc"
+#undef DECL_INTRINSIC_RECORDS
+};
+
+static ArrayRef<TypeProfile> ProtoSeq2ArrayRef(uint16_t Index, uint8_t Length) {
+ return ArrayRef<TypeProfile>(&RVVSignatureTable[Index], Length);
+}
+
+static QualType RVVType2Qual(ASTContext &Context, const RVVType *Type) {
+ QualType QT;
+ switch (Type->getScalarType()) {
+ case ScalarTypeKind::Void:
+ QT = Context.VoidTy;
+ break;
+ case ScalarTypeKind::Size_t:
+ QT = Context.getSizeType();
+ break;
+ case ScalarTypeKind::Ptr
diff _t:
+ QT = Context.getPointerDiffType();
+ break;
+ case ScalarTypeKind::UnsignedLong:
+ QT = Context.UnsignedLongTy;
+ break;
+ case ScalarTypeKind::SignedLong:
+ QT = Context.LongTy;
+ break;
+ case ScalarTypeKind::Boolean:
+ QT = Context.BoolTy;
+ break;
+ case ScalarTypeKind::SignedInteger:
+ QT = Context.getIntTypeForBitwidth(Type->getElementBitwidth(), true);
+ break;
+ case ScalarTypeKind::UnsignedInteger:
+ QT = Context.getIntTypeForBitwidth(Type->getElementBitwidth(), false);
+ break;
+ case ScalarTypeKind::Float:
+ switch (Type->getElementBitwidth()) {
+ case 64:
+ QT = Context.DoubleTy;
+ break;
+ case 32:
+ QT = Context.FloatTy;
+ break;
+ case 16:
+ QT = Context.Float16Ty;
+ break;
+ }
+ break;
+ default:
+ return QT;
+ }
+ if (Type->isVector())
+ QT = Context.getScalableVectorType(QT, Type->getScale().getValue());
+
+ if (Type->isConstant()) {
+ QT = Context.getConstType(QT);
+ }
+
+ // Transform the type to a pointer as the last step, if necessary.
+ if (Type->isPointer()) {
+ QT = Context.getPointerType(QT);
+ }
+
+ return QT;
+}
+
+static void
+CreateRVVIntrinsicDecl(Sema &S, LookupResult &LR, IdentifierInfo *II,
+ Preprocessor &PP, unsigned Index,
+ std::vector<RVVIntrinsicDef> &RVVIntrinsicList,
+ bool IsOverload) {
+ ASTContext &Context = S.Context;
+ RVVIntrinsicDef &IDef = RVVIntrinsicList[Index];
+ auto Sigs = IDef.Signature;
+ size_t SigLength = Sigs.size();
+ auto ReturnType = Sigs[0];
+ QualType RetType = RVVType2Qual(Context, ReturnType);
+ SmallVector<QualType, 8> ArgTypes;
+ QualType BuiltinFuncType;
+
+ // Skip return type, and convert RVVType to QualType for arguments.
+ for (size_t i = 1; i < SigLength; ++i) {
+ ArgTypes.push_back(RVVType2Qual(Context, Sigs[i]));
+ }
+
+ FunctionProtoType::ExtProtoInfo PI(
+ Context.getDefaultCallingConvention(false, false, true));
+
+ PI.Variadic = false;
+
+ SourceLocation Loc = LR.getNameLoc();
+ BuiltinFuncType = Context.getFunctionType(RetType, ArgTypes, PI);
+ DeclContext *Parent = Context.getTranslationUnitDecl();
+
+ FunctionDecl *RVVIntrinsicDecl = FunctionDecl::Create(
+ Context, Parent, Loc, Loc, II, BuiltinFuncType, /*TInfo=*/nullptr,
+ SC_Extern, S.getCurFPFeatures().isFPConstrained(), false,
+ BuiltinFuncType->isFunctionProtoType());
+
+ // Create Decl objects for each parameter, adding them to the
+ // FunctionDecl.
+ const auto *FP = cast<FunctionProtoType>(BuiltinFuncType);
+ SmallVector<ParmVarDecl *, 8> ParmList;
+ for (unsigned IParm = 0, e = FP->getNumParams(); IParm != e; ++IParm) {
+ ParmVarDecl *Parm =
+ ParmVarDecl::Create(Context, RVVIntrinsicDecl, Loc, Loc, nullptr,
+ FP->getParamType(IParm), nullptr, SC_None, nullptr);
+ Parm->setScopeInfo(0, IParm);
+ ParmList.push_back(Parm);
+ }
+ RVVIntrinsicDecl->setParams(ParmList);
+
+ // Add function attributes.
+ if (IsOverload)
+ RVVIntrinsicDecl->addAttr(OverloadableAttr::CreateImplicit(Context));
+
+ // Setup alias to __builtin_rvv_*
+ auto &IntrinsicII = PP.getIdentifierTable().get(IDef.BuiltinName);
+ RVVIntrinsicDecl->addAttr(
+ BuiltinAliasAttr::CreateImplicit(S.Context, &IntrinsicII));
+
+ // Add to symbol table.
+ LR.addDecl(RVVIntrinsicDecl);
+}
+
+namespace {
+class RVVIntrinsicManager {
+private:
+ // List of all RVV intrinsic.
+ std::vector<RVVIntrinsicDef> IntrinsicList;
+ // Mapping function name to index of IntrinsicList.
+ StringMap<size_t> Intrinsics;
+ // Mapping function name to RVVOverloadIntrinsicDef.
+ StringMap<RVVOverloadIntrinsicDef> OverloadIntrinsics;
+
+ ASTContext &Context;
+
+ // Create IntrinsicList
+ void InitIntrinsicList();
+
+ // Create RVVIntrinsicDef.
+ void InitRVVIntrinsic(const RVVIntrinsicRecord &Record, StringRef SuffixStr,
+ StringRef OverloadedSuffixStr, bool IsMask,
+ RVVTypes &Types);
+
+public:
+ RVVIntrinsicManager(ASTContext &Context) : Context(Context) {
+ InitIntrinsicList();
+ }
+
+ bool CreateIntrinsicIfFound(Sema &S, LookupResult &LR, IdentifierInfo *II,
+ Preprocessor &PP);
+};
+} // namespace
+
+void RVVIntrinsicManager::InitIntrinsicList() {
+ const TargetInfo &TI = Context.getTargetInfo();
+ bool HasVectorFloat32 = TI.hasFeature("zve32f");
+ bool HasVectorFloat64 = TI.hasFeature("zve64d");
+ bool HasZvfh = TI.hasFeature("experimental-zvfh");
+ bool HasRV64 = TI.hasFeature("64bit");
+ bool HasFullMultiply = TI.hasFeature("v");
+
+ for (auto &Record : RVVIntrinsicRecords) {
+ // Create Intrinsics for each type and LMUL.
+ BasicType BaseType = BasicType::Unknown;
+ auto ProtoSeq =
+ ProtoSeq2ArrayRef(Record.PrototypeIndex, Record.PrototypeLength);
+ auto ProtoMaskSeq = ProtoSeq2ArrayRef(Record.MaskedPrototypeIndex,
+ Record.MaskedPrototypeLength);
+ auto SuffixProto =
+ ProtoSeq2ArrayRef(Record.SuffixIndex, Record.SuffixLength);
+ auto MangledSuffixProto = ProtoSeq2ArrayRef(Record.OverloadedSuffixIndex,
+ Record.OverloadedSuffixSize);
+ for (int TypeRangeMaskShift = 0;
+ TypeRangeMaskShift <= static_cast<int>(BasicType::MaxOffset);
+ ++TypeRangeMaskShift) {
+ int BaseTypeI = 1 << TypeRangeMaskShift;
+ BaseType = static_cast<BasicType>(BaseTypeI);
+
+ if ((BaseTypeI & Record.TypeRangeMask) != BaseTypeI)
+ continue;
+
+ // Check requirement.
+ if (BaseType == BasicType::Float16 && !HasZvfh)
+ continue;
+
+ if (BaseType == BasicType::Float32 && !HasVectorFloat32)
+ continue;
+
+ if (BaseType == BasicType::Float64 && !HasVectorFloat64)
+ continue;
+
+ if (((Record.RequiredExtension & RVVRequire::RV64) == RVVRequire::RV64) &&
+ !HasRV64)
+ continue;
+
+ if ((BaseType == BasicType::Int64) &&
+ ((Record.RequiredExtension & RVVRequire::FullMultiply) ==
+ RVVRequire::FullMultiply) &&
+ !HasFullMultiply)
+ continue;
+
+ for (int Log2LMUL = -3; Log2LMUL <= 3; Log2LMUL++) {
+ if (!(Record.Log2LMULMask & (1 << (Log2LMUL + 3)))) {
+ continue;
+ }
+ Optional<RVVTypes> Types =
+ RVVType::computeTypes(BaseType, Log2LMUL, Record.NF, ProtoSeq);
+
+ // Ignored to create new intrinsic if there are any illegal types.
+ if (!Types.hasValue()) {
+ continue;
+ }
+
+ auto SuffixStr =
+ RVVIntrinsic::getSuffixStr(BaseType, Log2LMUL, SuffixProto);
+ auto OverloadedSuffixStr =
+ RVVIntrinsic::getSuffixStr(BaseType, Log2LMUL, MangledSuffixProto);
+
+ // Create non-masked intrinsic.
+ InitRVVIntrinsic(Record, SuffixStr, OverloadedSuffixStr, false, *Types);
+
+ bool HasMask = Record.MaskedPrototypeLength != 0;
+
+ if (HasMask) {
+ // Create masked intrinsic.
+ Optional<RVVTypes> MaskTypes = RVVType::computeTypes(
+ BaseType, Log2LMUL, Record.NF, ProtoMaskSeq);
+
+ InitRVVIntrinsic(Record, SuffixStr, OverloadedSuffixStr, true,
+ *MaskTypes);
+ }
+ }
+ }
+ }
+}
+
+// Compute name and signatures for intrinsic with practical types.
+void RVVIntrinsicManager::InitRVVIntrinsic(const RVVIntrinsicRecord &Record,
+ StringRef SuffixStr,
+ StringRef OverloadedSuffixStr,
+ bool IsMask, RVVTypes &Signature) {
+ // Function name, e.g. vadd_vv_i32m1.
+ std::string Name = Record.Name;
+ if (!SuffixStr.empty())
+ Name += "_" + SuffixStr.str();
+
+ if (IsMask) {
+ Name += "_m";
+ }
+
+ // Overloaded function name, e.g. vadd.
+ std::string OverloadedName;
+ if (!Record.OverloadedName)
+ OverloadedName = StringRef(Record.Name).split("_").first.str();
+ else
+ OverloadedName = Record.OverloadedName;
+ if (!OverloadedSuffixStr.empty())
+ OverloadedName += "_" + OverloadedSuffixStr.str();
+
+ // clang built-in function name, e.g. __builtin_rvv_vadd.
+ std::string BuiltinName = "__builtin_rvv_" + std::string(Record.Name);
+ if (IsMask) {
+ BuiltinName += "_m";
+ }
+
+ // Put into IntrinsicList.
+ size_t Index = IntrinsicList.size();
+ IntrinsicList.push_back({Name, OverloadedName, BuiltinName, Signature});
+
+ // Creating mapping to Intrinsics.
+ Intrinsics.insert({Name, Index});
+
+ // Get the RVVOverloadIntrinsicDef.
+ RVVOverloadIntrinsicDef &OverloadIntrinsicDef =
+ OverloadIntrinsics[OverloadedName];
+
+ // And added the index.
+ OverloadIntrinsicDef.Indexs.push_back(Index);
+}
+
+bool RVVIntrinsicManager::CreateIntrinsicIfFound(Sema &S, LookupResult &LR,
+ IdentifierInfo *II,
+ Preprocessor &PP) {
+ StringRef Name = II->getName();
+
+ // Lookup the function name from the overload intrinsics first.
+ auto OvIItr = OverloadIntrinsics.find(Name);
+ if (OvIItr != OverloadIntrinsics.end()) {
+ auto OvIntrinsicDef = OvIItr->second;
+ for (auto Index : OvIntrinsicDef.Indexs) {
+ CreateRVVIntrinsicDecl(S, LR, II, PP, Index, IntrinsicList,
+ /*IsOverload*/ true);
+ }
+
+ // If we added overloads, need to resolve the lookup result.
+ LR.resolveKind();
+ return true;
+ }
+
+ // Lookup the function name from the intrinsics.
+ auto Itr = Intrinsics.find(Name);
+ if (Itr != Intrinsics.end()) {
+ CreateRVVIntrinsicDecl(S, LR, II, PP, Itr->second, IntrinsicList,
+ /*IsOverload*/ false);
+ return true;
+ }
+
+ // It's not RVV intrinsics.
+ return false;
+}
+
+namespace clang {
+bool GetRVVBuiltinInfo(Sema &S, LookupResult &LR, IdentifierInfo *II,
+ Preprocessor &PP) {
+ static std::unique_ptr<RVVIntrinsicManager> IntrinsicManager =
+ std::make_unique<RVVIntrinsicManager>(S.Context);
+
+ return IntrinsicManager->CreateIntrinsicIfFound(S, LR, II, PP);
+}
+} // namespace clang
diff --git a/clang/lib/Support/RISCVVIntrinsicUtils.cpp b/clang/lib/Support/RISCVVIntrinsicUtils.cpp
index 0f21aa113eec3..771a7fd2ddb6a 100644
--- a/clang/lib/Support/RISCVVIntrinsicUtils.cpp
+++ b/clang/lib/Support/RISCVVIntrinsicUtils.cpp
@@ -13,6 +13,7 @@
#include "llvm/ADT/StringExtras.h"
#include "llvm/ADT/StringMap.h"
#include "llvm/ADT/StringSet.h"
+#include "llvm/ADT/StringSwitch.h"
#include "llvm/ADT/Twine.h"
#include "llvm/Support/raw_ostream.h"
#include <numeric>
@@ -853,27 +854,6 @@ RVVIntrinsic::RVVIntrinsic(
Name += "_m";
}
- // Init RISC-V extensions
- for (const auto &T : OutInTypes) {
- if (T->isFloatVector(16) || T->isFloat(16))
- RISCVPredefinedMacros |= RISCVPredefinedMacro::Zvfh;
- if (T->isFloatVector(32))
- RISCVPredefinedMacros |= RISCVPredefinedMacro::VectorMaxELenFp32;
- if (T->isFloatVector(64))
- RISCVPredefinedMacros |= RISCVPredefinedMacro::VectorMaxELenFp64;
- if (T->isVector(64))
- RISCVPredefinedMacros |= RISCVPredefinedMacro::VectorMaxELen64;
- }
- for (auto Feature : RequiredFeatures) {
- if (Feature == "RV64")
- RISCVPredefinedMacros |= RISCVPredefinedMacro::RV64;
- // Note: Full multiply instruction (mulh, mulhu, mulhsu, smul) for EEW=64
- // require V.
- if (Feature == "FullMultiply" &&
- (RISCVPredefinedMacros & RISCVPredefinedMacro::VectorMaxELen64))
- RISCVPredefinedMacros |= RISCVPredefinedMacro::V;
- }
-
// Init OutputType and InputTypes
OutputType = OutInTypes[0];
InputTypes.assign(OutInTypes.begin() + 1, OutInTypes.end());
@@ -901,7 +881,7 @@ std::string RVVIntrinsic::getBuiltinTypeStr() const {
std::string
RVVIntrinsic::getSuffixStr(BasicType Type, int Log2LMUL,
- const llvm::SmallVector<TypeProfile> &TypeProfiles) {
+ const llvm::ArrayRef<TypeProfile> &TypeProfiles) {
SmallVector<std::string> SuffixStrs;
for (auto TP : TypeProfiles) {
auto T = RVVType::computeType(Type, Log2LMUL, TP);
diff --git a/clang/utils/TableGen/RISCVVEmitter.cpp b/clang/utils/TableGen/RISCVVEmitter.cpp
index e8083d99f4742..139ddc17f1a82 100644
--- a/clang/utils/TableGen/RISCVVEmitter.cpp
+++ b/clang/utils/TableGen/RISCVVEmitter.cpp
@@ -20,6 +20,7 @@
#include "llvm/ADT/StringExtras.h"
#include "llvm/ADT/StringMap.h"
#include "llvm/ADT/StringSet.h"
+#include "llvm/ADT/StringSwitch.h"
#include "llvm/ADT/Twine.h"
#include "llvm/TableGen/Error.h"
#include "llvm/TableGen/Record.h"
@@ -29,9 +30,48 @@ using namespace llvm;
using namespace clang::RISCV;
namespace {
+struct SemaRecord {
+ // Intrinsic name, e.g. vadd_vv
+ std::string Name;
+
+ // Overloaded intrinsic name, could be empty if can be computed from Name
+ // e.g. vadd
+ std::string OverloadedName;
+
+ // Supported type, mask of BasicType
+ unsigned TypeRangeMask;
+
+ // Supported LMUL.
+ unsigned Log2LMULMask;
+
+ // Required target features for this intrinsic.
+ std::vector<std::string> RequiredFeatures;
+
+ // Prototype for this intrinsic.
+ SmallVector<TypeProfile> Prototype;
+
+ // Prototype for masked intrinsic.
+ SmallVector<TypeProfile> MaskedPrototype;
+
+ // Suffix of intrinsic name.
+ SmallVector<TypeProfile> Suffix;
+
+ // Suffix of overloaded intrinsic name.
+ SmallVector<TypeProfile> OverloadedSuffix;
+
+ // Number of field, large than 1 if it's segment load/store.
+ unsigned NF;
+};
+
class RVVEmitter {
private:
RecordKeeper &Records;
+ // Concat BasicType, LMUL and Proto as key
+ StringMap<RVVType> LegalTypes;
+ StringSet<> IllegalTypes;
+
+ std::vector<SemaRecord> SemaRecords;
+ std::vector<TypeProfile> SemaSignatureTable;
public:
RVVEmitter(RecordKeeper &R) : Records(R) {}
@@ -45,22 +85,27 @@ class RVVEmitter {
/// Emit all the information needed to map builtin -> LLVM IR intrinsic.
void createCodeGen(raw_ostream &o);
+ /// Emit all the information needed by SemaRVVLookup.cpp.
+ /// We've large number of intrinsic function for RVV, creating a customized
+ /// could speed up the compilation time.
+ void createSema(raw_ostream &o);
+
private:
/// Create all intrinsics and add them to \p Out
void createRVVIntrinsics(std::vector<std::unique_ptr<RVVIntrinsic>> &Out);
/// Print HeaderCode in RVVHeader Record to \p Out
void printHeaderCode(raw_ostream &OS);
- /// Emit Acrh predecessor definitions and body, assume the element of Defs are
- /// sorted by extension.
- void emitArchMacroAndBody(
- std::vector<std::unique_ptr<RVVIntrinsic>> &Defs, raw_ostream &o,
- std::function<void(raw_ostream &, const RVVIntrinsic &)>);
+ /// Construct a compressed signature table used for createSema.
+ void ConstructSemaSignatureTable();
+ /// Get index from the signature table.
+ unsigned GetSemaSignatureIndex(const SmallVector<TypeProfile> &Signature);
+
+ /// Emit all SemaRecord for SemaRVVLookup.cpp.
+ void EmitSemaRecords(raw_ostream &OS);
- // Emit the architecture preprocessor definitions. Return true when emits
- // non-empty string.
- bool emitMacroRestrictionStr(RISCVPredefinedMacroT PredefinedMacros,
- raw_ostream &o);
+ /// Emit signature table for SemaRVVLookup.cpp.
+ void EmitSemaSignatureTable(raw_ostream &OS);
};
} // namespace
@@ -141,35 +186,6 @@ void emitCodeGenSwitchBody(const RVVIntrinsic *RVVI, raw_ostream &OS) {
OS << " break;\n";
}
-void emitIntrinsicFuncDef(const RVVIntrinsic &RVVI, raw_ostream &OS) {
- OS << "__attribute__((__clang_builtin_alias__(";
- OS << "__builtin_rvv_" << RVVI.getBuiltinName() << ")))\n";
- OS << RVVI.getOutputType()->getTypeStr() << " " << RVVI.getName() << "(";
- // Emit function arguments
- const RVVTypes &InputTypes = RVVI.getInputTypes();
- if (!InputTypes.empty()) {
- ListSeparator LS;
- for (unsigned i = 0; i < InputTypes.size(); ++i)
- OS << LS << InputTypes[i]->getTypeStr();
- }
- OS << ");\n";
-}
-
-void emitMangledFuncDef(const RVVIntrinsic &RVVI, raw_ostream &OS) {
- OS << "__attribute__((__clang_builtin_alias__(";
- OS << "__builtin_rvv_" << RVVI.getBuiltinName() << ")))\n";
- OS << RVVI.getOutputType()->getTypeStr() << " " << RVVI.getMangledName()
- << "(";
- // Emit function arguments
- const RVVTypes &InputTypes = RVVI.getInputTypes();
- if (!InputTypes.empty()) {
- ListSeparator LS;
- for (unsigned i = 0; i < InputTypes.size(); ++i)
- OS << LS << InputTypes[i]->getTypeStr();
- }
- OS << ");\n";
-}
-
//===----------------------------------------------------------------------===//
// RVVEmitter implementation
//===----------------------------------------------------------------------===//
@@ -202,10 +218,9 @@ void RVVEmitter::createHeader(raw_ostream &OS) {
OS << "extern \"C\" {\n";
OS << "#endif\n\n";
- printHeaderCode(OS);
+ OS << "#pragma clang riscv intrinsic vector\n\n";
- std::vector<std::unique_ptr<RVVIntrinsic>> Defs;
- createRVVIntrinsics(Defs);
+ printHeaderCode(OS);
auto printType = [&](auto T) {
OS << "typedef " << T->getClangBuiltinStr() << " " << T->getTypeStr()
@@ -260,37 +275,8 @@ void RVVEmitter::createHeader(raw_ostream &OS) {
}
OS << "#endif\n\n";
- // The same extension include in the same arch guard marco.
- llvm::stable_sort(Defs, [](const std::unique_ptr<RVVIntrinsic> &A,
- const std::unique_ptr<RVVIntrinsic> &B) {
- return A->getRISCVPredefinedMacros() < B->getRISCVPredefinedMacros();
- });
-
- OS << "#define __rvv_ai static __inline__\n";
-
- // Print intrinsic functions with macro
- emitArchMacroAndBody(Defs, OS, [](raw_ostream &OS, const RVVIntrinsic &Inst) {
- OS << "__rvv_ai ";
- emitIntrinsicFuncDef(Inst, OS);
- });
-
- OS << "#undef __rvv_ai\n\n";
-
OS << "#define __riscv_v_intrinsic_overloading 1\n";
- // Print Overloaded APIs
- OS << "#define __rvv_aio static __inline__ "
- "__attribute__((__overloadable__))\n";
-
- emitArchMacroAndBody(Defs, OS, [](raw_ostream &OS, const RVVIntrinsic &Inst) {
- if (!Inst.isMasked() && !Inst.hasUnMaskedOverloaded())
- return;
- OS << "__rvv_aio ";
- emitMangledFuncDef(Inst, OS);
- });
-
- OS << "#undef __rvv_aio\n";
-
OS << "\n#ifdef __cplusplus\n";
OS << "}\n";
OS << "#endif // __cplusplus\n";
@@ -383,9 +369,9 @@ void RVVEmitter::createRVVIntrinsics(
std::vector<Record *> RV = Records.getAllDerivedDefinitions("RVVBuiltin");
for (auto *R : RV) {
StringRef Name = R->getValueAsString("Name");
- StringRef SuffixProto = R->getValueAsString("Suffix");
+ StringRef Suffix = R->getValueAsString("Suffix");
StringRef MangledName = R->getValueAsString("MangledName");
- StringRef MangledSuffixProto = R->getValueAsString("MangledSuffix");
+ StringRef OverloadedSuffix = R->getValueAsString("MangledSuffix");
StringRef Prototypes = R->getValueAsString("Prototype");
StringRef TypeRange = R->getValueAsString("TypeRange");
bool HasMasked = R->getValueAsBit("HasMasked");
@@ -411,29 +397,30 @@ void RVVEmitter::createRVVIntrinsics(
unsigned NF = R->getValueAsInt("NF");
// Parse prototype and create a list of primitive type with transformers
- // (operand) in ProtoSeq. ProtoSeq[0] is output operand.
- SmallVector<TypeProfile> ProtoSeq = parsePrototypes(Prototypes);
+ // (operand) in Prototype. Prototype[0] is output operand.
+ SmallVector<TypeProfile> Prototype = parsePrototypes(Prototypes);
- SmallVector<TypeProfile> SuffixProtoSeq = parsePrototypes(SuffixProto);
- SmallVector<TypeProfile> MangledSuffixProtoSeq =
- parsePrototypes(MangledSuffixProto);
+ SmallVector<TypeProfile> Suffixtype = parsePrototypes(Suffix);
+ SmallVector<TypeProfile> OverloadedSuffixtype =
+ parsePrototypes(OverloadedSuffix);
// Compute Builtin types
- SmallVector<TypeProfile> ProtoMaskSeq = ProtoSeq;
+ SmallVector<TypeProfile> MaskedPrototype = Prototype;
if (HasMasked) {
// If HasMaskedOffOperand, insert result type as first input operand.
if (HasMaskedOffOperand) {
if (NF == 1) {
- ProtoMaskSeq.insert(ProtoMaskSeq.begin() + 1, ProtoSeq[0]);
+ MaskedPrototype.insert(MaskedPrototype.begin() + 1, Prototype[0]);
} else {
// Convert
// (void, op0 address, op1 address, ...)
// to
// (void, op0 address, op1 address, ..., maskedoff0, maskedoff1, ...)
- TypeProfile MaskoffType = ProtoSeq[1];
+ TypeProfile MaskoffType = Prototype[1];
MaskoffType.TM &= ~static_cast<uint8_t>(TypeModifier::Pointer);
for (unsigned I = 0; I < NF; ++I)
- ProtoMaskSeq.insert(ProtoMaskSeq.begin() + NF + 1, MaskoffType);
+ MaskedPrototype.insert(MaskedPrototype.begin() + NF + 1,
+ MaskoffType);
}
}
if (HasMaskedOffOperand && NF > 1) {
@@ -442,16 +429,17 @@ void RVVEmitter::createRVVIntrinsics(
// to
// (void, op0 address, op1 address, ..., mask, maskedoff0, maskedoff1,
// ...)
- ProtoMaskSeq.insert(ProtoMaskSeq.begin() + NF + 1, TypeProfile::Mask);
+ MaskedPrototype.insert(MaskedPrototype.begin() + NF + 1,
+ TypeProfile::Mask);
} else {
// If HasMasked, insert TypeProfile:Mask as first input operand.
- ProtoMaskSeq.insert(ProtoMaskSeq.begin() + 1, TypeProfile::Mask);
+ MaskedPrototype.insert(MaskedPrototype.begin() + 1, TypeProfile::Mask);
}
}
// If HasVL, append TypeProfile:VL to last operand
if (HasVL) {
- ProtoSeq.push_back(TypeProfile::VL);
- ProtoMaskSeq.push_back(TypeProfile::VL);
+ Prototype.push_back(TypeProfile::VL);
+ MaskedPrototype.push_back(TypeProfile::VL);
}
// Create Intrinsics for each type and LMUL.
@@ -459,15 +447,14 @@ void RVVEmitter::createRVVIntrinsics(
for (int Log2LMUL : Log2LMULList) {
BasicType BT = ParseBasicType(I);
Optional<RVVTypes> Types =
- RVVType::computeTypes(BT, Log2LMUL, NF, ProtoSeq);
+ RVVType::computeTypes(BT, Log2LMUL, NF, Prototype);
// Ignored to create new intrinsic if there are any illegal types.
if (!Types.hasValue())
continue;
- auto SuffixStr =
- RVVIntrinsic::getSuffixStr(BT, Log2LMUL, SuffixProtoSeq);
+ auto SuffixStr = RVVIntrinsic::getSuffixStr(BT, Log2LMUL, Suffixtype);
auto MangledSuffixStr =
- RVVIntrinsic::getSuffixStr(BT, Log2LMUL, MangledSuffixProtoSeq);
+ RVVIntrinsic::getSuffixStr(BT, Log2LMUL, OverloadedSuffixtype);
// Create a unmasked intrinsic
Out.push_back(std::make_unique<RVVIntrinsic>(
Name, SuffixStr, MangledName, MangledSuffixStr, IRName,
@@ -478,7 +465,7 @@ void RVVEmitter::createRVVIntrinsics(
if (HasMasked) {
// Create a masked intrinsic
Optional<RVVTypes> MaskTypes =
- RVVType::computeTypes(BT, Log2LMUL, NF, ProtoMaskSeq);
+ RVVType::computeTypes(BT, Log2LMUL, NF, MaskedPrototype);
Out.push_back(std::make_unique<RVVIntrinsic>(
Name, SuffixStr, MangledName, MangledSuffixStr, MaskedIRName,
/*IsMasked=*/true, HasMaskedOffOperand, HasVL, MaskedPolicy,
@@ -487,6 +474,46 @@ void RVVEmitter::createRVVIntrinsics(
}
} // end for Log2LMULList
} // end for TypeRange
+
+ // We don't emit vsetvli and vsetvlimax for SemaRecord.
+ // They are handled by riscv_vector.h
+ if (Name == "vsetvli" || Name == "vsetvlimax")
+ continue;
+ // Create SemaRecord
+ SemaRecord SR;
+ SR.Name = Name.str();
+ SR.OverloadedName = MangledName.str();
+ BasicType TypeRangeMask = BasicType::Unknown;
+ for (char I : TypeRange) {
+ TypeRangeMask |= ParseBasicType(I);
+ }
+ SR.TypeRangeMask = static_cast<unsigned>(TypeRangeMask);
+
+ unsigned Log2LMULMask = 0;
+ for (int Log2LMUL : Log2LMULList) {
+ Log2LMULMask |= 1 << (Log2LMUL + 3);
+ }
+
+ SR.Log2LMULMask = Log2LMULMask;
+ for (auto RequiredFeature : RequiredFeatures)
+ SR.RequiredFeatures.push_back(RequiredFeature.str());
+
+ SR.NF = NF;
+
+ SR.Prototype = std::move(Prototype);
+
+ if (HasMasked)
+ SR.MaskedPrototype = std::move(MaskedPrototype);
+
+ auto InitSuffixtype = [&](SmallVectorImpl<TypeProfile> &PS,
+ StringRef Prototypes) {
+ PS = parsePrototypes(Prototypes);
+ };
+
+ InitSuffixtype(SR.Suffix, Suffix);
+ InitSuffixtype(SR.OverloadedSuffix, OverloadedSuffix);
+
+ SemaRecords.push_back(SR);
}
}
@@ -499,47 +526,131 @@ void RVVEmitter::printHeaderCode(raw_ostream &OS) {
}
}
-void RVVEmitter::emitArchMacroAndBody(
- std::vector<std::unique_ptr<RVVIntrinsic>> &Defs, raw_ostream &OS,
- std::function<void(raw_ostream &, const RVVIntrinsic &)> PrintBody) {
- RISCVPredefinedMacroT PrevMacros =
- (*Defs.begin())->getRISCVPredefinedMacros();
- bool NeedEndif = emitMacroRestrictionStr(PrevMacros, OS);
- for (auto &Def : Defs) {
- RISCVPredefinedMacroT CurMacros = Def->getRISCVPredefinedMacros();
- if (CurMacros != PrevMacros) {
- if (NeedEndif)
- OS << "#endif\n\n";
- NeedEndif = emitMacroRestrictionStr(CurMacros, OS);
- PrevMacros = CurMacros;
+unsigned
+RVVEmitter::GetSemaSignatureIndex(const SmallVector<TypeProfile> &Signature) {
+ if (Signature.size() == 0)
+ return 0;
+
+ // Checking Signature already in table or not.
+ if (Signature.size() < SemaSignatureTable.size()) {
+ size_t Bound = SemaSignatureTable.size() - Signature.size() + 1;
+ for (size_t Index = 0; Index < Bound; ++Index) {
+ bool Match = true;
+ for (size_t i = 0; i < Signature.size(); ++i) {
+ if (Signature[i] != SemaSignatureTable[Index + i]) {
+ Match = false;
+ break;
+ }
+ }
+ // Reuse if found in table.
+ if (Match)
+ return Index;
}
- if (Def->hasBuiltinAlias())
- PrintBody(OS, *Def);
}
- if (NeedEndif)
- OS << "#endif\n\n";
+
+ // Insert Signature into SemaSignatureTable if not found in the table.
+ size_t Index = SemaSignatureTable.size();
+ for (const auto &Type : Signature) {
+ SemaSignatureTable.push_back(Type);
+ }
+ return Index;
}
-bool RVVEmitter::emitMacroRestrictionStr(RISCVPredefinedMacroT PredefinedMacros,
- raw_ostream &OS) {
- if (PredefinedMacros == RISCVPredefinedMacro::Basic)
- return false;
- OS << "#if ";
- ListSeparator LS(" && ");
- if (PredefinedMacros & RISCVPredefinedMacro::V)
- OS << LS << "defined(__riscv_v)";
- if (PredefinedMacros & RISCVPredefinedMacro::Zvfh)
- OS << LS << "defined(__riscv_zvfh)";
- if (PredefinedMacros & RISCVPredefinedMacro::RV64)
- OS << LS << "(__riscv_xlen == 64)";
- if (PredefinedMacros & RISCVPredefinedMacro::VectorMaxELen64)
- OS << LS << "(__riscv_v_elen >= 64)";
- if (PredefinedMacros & RISCVPredefinedMacro::VectorMaxELenFp32)
- OS << LS << "(__riscv_v_elen_fp >= 32)";
- if (PredefinedMacros & RISCVPredefinedMacro::VectorMaxELenFp64)
- OS << LS << "(__riscv_v_elen_fp >= 64)";
- OS << "\n";
- return true;
+void RVVEmitter::ConstructSemaSignatureTable() {
+ // Sort signature entries by length, let longer signature insert first, to
+ // make it more possible to reuse table entries, that can reduce ~10% table
+ // size.
+ struct Compare {
+ bool operator()(const SmallVector<TypeProfile> &A,
+ const SmallVector<TypeProfile> &B) {
+ if (A.size() != B.size())
+ return A.size() > B.size();
+
+ size_t Len = A.size();
+ for (size_t i = 0; i < Len; ++i) {
+ if (A[i] != B[i])
+ return A[i] > B[i];
+ }
+
+ return false;
+ }
+ };
+
+ std::set<SmallVector<TypeProfile>, Compare> Signatures;
+ auto InsertToSignatureSet = [&](const SmallVector<TypeProfile> &Signature) {
+ if (Signature.empty())
+ return;
+
+ Signatures.insert(Signature);
+ };
+
+ for (const auto &SemaRecord : SemaRecords) {
+ InsertToSignatureSet(SemaRecord.Prototype);
+ InsertToSignatureSet(SemaRecord.MaskedPrototype);
+ InsertToSignatureSet(SemaRecord.Suffix);
+ InsertToSignatureSet(SemaRecord.OverloadedSuffix);
+ }
+
+ for (const auto &Sig : Signatures) {
+ GetSemaSignatureIndex(Sig);
+ }
+}
+
+void RVVEmitter::EmitSemaSignatureTable(raw_ostream &OS) {
+ OS << "#ifdef DECL_SIGNATURE_TABLE\n";
+ for (const auto &Sig : SemaSignatureTable)
+ OS << "TypeProfile(" << static_cast<int>(Sig.PT) << ", "
+ << static_cast<int>(Sig.VTM) << ", " << static_cast<int>(Sig.TM)
+ << "),\n";
+ OS << "#endif\n";
+}
+
+void RVVEmitter::EmitSemaRecords(raw_ostream &OS) {
+ OS << "#ifdef DECL_INTRINSIC_RECORDS\n";
+ for (const auto &SR : SemaRecords) {
+ // Output *MUST* sync with RVVIntrinsicRecord in SemaRVVLookup.cpp.
+ OS << "{"
+ << "\"" << SR.Name << "\", ";
+
+ if (SR.OverloadedName.empty())
+ OS << "nullptr, ";
+ else
+ OS << "\"" << SR.OverloadedName << "\", ";
+
+ OS << GetSemaSignatureIndex(SR.Prototype) << ", ";
+ OS << GetSemaSignatureIndex(SR.MaskedPrototype) << ", ";
+ OS << GetSemaSignatureIndex(SR.Suffix) << ", ";
+ OS << GetSemaSignatureIndex(SR.OverloadedSuffix) << ", ";
+
+ OS << SR.Prototype.size() << ", ";
+ OS << SR.MaskedPrototype.size() << ", ";
+ OS << SR.Suffix.size() << ", ";
+ OS << SR.OverloadedSuffix.size() << ", ";
+
+ if (SR.RequiredFeatures.empty())
+ OS << "RVVRequire::None";
+ else {
+ ListSeparator LS(" | ");
+ for (auto RequiredFeature : SR.RequiredFeatures)
+ OS << LS << "RVVRequire::" << RequiredFeature;
+ }
+ OS << ",";
+
+ OS << " /* Type Range Mask*/" << SR.TypeRangeMask << ", ";
+ OS << " /* LMUL Mask = */ " << SR.Log2LMULMask << ", ";
+
+ OS << SR.NF << "},\n";
+ }
+ OS << "#endif\n";
+}
+
+void RVVEmitter::createSema(raw_ostream &OS) {
+ std::vector<std::unique_ptr<RVVIntrinsic>> Defs;
+ createRVVIntrinsics(Defs);
+
+ ConstructSemaSignatureTable();
+ EmitSemaSignatureTable(OS);
+ EmitSemaRecords(OS);
}
namespace clang {
@@ -555,4 +666,8 @@ void EmitRVVBuiltinCG(RecordKeeper &Records, raw_ostream &OS) {
RVVEmitter(Records).createCodeGen(OS);
}
+void EmitRVVBuiltinSema(RecordKeeper &Records, raw_ostream &OS) {
+ RVVEmitter(Records).createSema(OS);
+}
+
} // End namespace clang
diff --git a/clang/utils/TableGen/TableGen.cpp b/clang/utils/TableGen/TableGen.cpp
index bb9366e2b7fc0..d18a31226e80d 100644
--- a/clang/utils/TableGen/TableGen.cpp
+++ b/clang/utils/TableGen/TableGen.cpp
@@ -88,6 +88,7 @@ enum ActionType {
GenRISCVVectorHeader,
GenRISCVVectorBuiltins,
GenRISCVVectorBuiltinCG,
+ GenRISCVVectorBuiltinSema,
GenAttrDocs,
GenDiagDocs,
GenOptDocs,
@@ -243,6 +244,8 @@ cl::opt<ActionType> Action(
"Generate riscv_vector_builtins.inc for clang"),
clEnumValN(GenRISCVVectorBuiltinCG, "gen-riscv-vector-builtin-codegen",
"Generate riscv_vector_builtin_cg.inc for clang"),
+ clEnumValN(GenRISCVVectorBuiltinSema, "gen-riscv-vector-builtin-sema",
+ "Generate riscv_vector_builtin_sema.inc for clang"),
clEnumValN(GenAttrDocs, "gen-attr-docs",
"Generate attribute documentation"),
clEnumValN(GenDiagDocs, "gen-diag-docs",
@@ -458,6 +461,9 @@ bool ClangTableGenMain(raw_ostream &OS, RecordKeeper &Records) {
case GenRISCVVectorBuiltinCG:
EmitRVVBuiltinCG(Records, OS);
break;
+ case GenRISCVVectorBuiltinSema:
+ EmitRVVBuiltinSema(Records, OS);
+ break;
case GenAttrDocs:
EmitClangAttrDocs(Records, OS);
break;
diff --git a/clang/utils/TableGen/TableGenBackends.h b/clang/utils/TableGen/TableGenBackends.h
index fd8b9fcda20f0..2ba857f66f50b 100644
--- a/clang/utils/TableGen/TableGenBackends.h
+++ b/clang/utils/TableGen/TableGenBackends.h
@@ -110,6 +110,7 @@ void EmitMveBuiltinAliases(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
void EmitRVVHeader(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
void EmitRVVBuiltins(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
void EmitRVVBuiltinCG(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
+void EmitRVVBuiltinSema(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
void EmitCdeHeader(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
void EmitCdeBuiltinDef(llvm::RecordKeeper &Records, llvm::raw_ostream &OS);
More information about the llvm-branch-commits
mailing list