[clang] [llvm] [RISCV] Inline Assembly Support for GPR Pairs ('Pr') (PR #112983)
Sam Elliott via cfe-commits
cfe-commits at lists.llvm.org
Wed Nov 13 06:05:56 PST 2024
https://github.com/lenary updated https://github.com/llvm/llvm-project/pull/112983
>From a63ead48585be1d6aa5e1b78926ede485e76e2f2 Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Tue, 22 Oct 2024 12:37:48 -0700
Subject: [PATCH 1/8] [RISCV] GPR Pairs for Inline Asm using `Pr`
This patch adds support for getting even-odd general purpose register
pairs into and out of inline assembly using the `Pr` constraint as
proposed in riscv-non-isa/riscv-c-api-doc#92
There are a few different pieces to this patch, each of which need their
own explanation.
Target-Independent Changes:
- This adds two new Machine Value Types (MVTs), which represent pairs for
each xlen. Two are needed because MVTs usually have a fixed length. This
change unfortunately increases the size of SelectionDAG tables indexed
by MVT by a small percentage.
- When an inline assembly block returns multiple values, it returns them
in a struct, rather than as a single value. This fixes TargetLowering
so that `getAsmOperandValueType` is called on the types in that
struct, so that targets have the opportunity to propose their own MVT
for an inline assembly operand where this wouldn't match conventional
arguments/return values. This matches what happens when a single value
is returned.
RISC-V Changes:
- Renames the Register Class used for f64 values on rv32i_zdinx from
`GPRPair*` to `GPRF64Pair*`. These register classes are kept broadly
unmodified, as their primary value type is used for type inference
over selection patterns. This rename affects quite a lot of files. I
reordered the definitions in RISCVRegisterInfo.td and added headings
to make it easier to browse.
- Adds new `GPRPair*` register classes which will be used for `Pr`
constraints and for instructions that need an even-odd GPR pair. This
new type is used for `amocas.d.*`(rv32) and `amocas.q.*`(rv64) in
Zacas, instead of the `GPRF64Pair` class being used before.
- Marks the new `GPRPair` class legal as for holding a
`MVT::riscv_i<xlen>_pair`. Two new RISCVISD node types are added for
creating and destructing a pair - `BuildGPRPair` and `SplitGPRPair`,
and are introduced when bitcasting to/from the pair type and the
`i<2*xlen>` type.
- This adds an override for `getNumRegisters` to ensure that `i<2*xlen>`
values, when going to/from inline assembly, only allocate one (pair)
register (they would otherwise allocate two).
- Ensures that the DAGCombiner doesn't merge the `bitcast` between
`i<2*xlen>` types and the pair type into a load/store, as we want to
legalise these 2*xlen-wide loads/stores as before - by splitting them
into two xlen-wide loads/stores, which will happen with `i<2*xlen>`
types.
- Ensures that Clang understands that `Pr` is a valid inline assembly
constraint.
---
clang/lib/Basic/Targets/RISCV.cpp | 11 +-
clang/test/CodeGen/RISCV/riscv-inline-asm.c | 13 ++
llvm/include/llvm/CodeGen/ValueTypes.td | 25 ++-
.../CodeGen/SelectionDAG/TargetLowering.cpp | 3 +-
llvm/lib/CodeGen/ValueTypes.cpp | 6 +
.../Target/RISCV/AsmParser/RISCVAsmParser.cpp | 22 +-
llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp | 31 ++-
llvm/lib/Target/RISCV/RISCVISelLowering.cpp | 81 +++++++-
llvm/lib/Target/RISCV/RISCVISelLowering.h | 24 +++
llvm/lib/Target/RISCV/RISCVInstrInfoD.td | 12 +-
llvm/lib/Target/RISCV/RISCVRegisterInfo.td | 195 +++++++++++-------
llvm/lib/Target/RISCV/RISCVSubtarget.h | 4 +
.../CodeGen/RISCV/rv32-inline-asm-pairs.ll | 73 +++++++
.../CodeGen/RISCV/rv64-inline-asm-pairs.ll | 73 +++++++
14 files changed, 459 insertions(+), 114 deletions(-)
create mode 100644 llvm/test/CodeGen/RISCV/rv32-inline-asm-pairs.ll
create mode 100644 llvm/test/CodeGen/RISCV/rv64-inline-asm-pairs.ll
diff --git a/clang/lib/Basic/Targets/RISCV.cpp b/clang/lib/Basic/Targets/RISCV.cpp
index eaaba7642bd7b2..07bf002ed73928 100644
--- a/clang/lib/Basic/Targets/RISCV.cpp
+++ b/clang/lib/Basic/Targets/RISCV.cpp
@@ -108,6 +108,14 @@ bool RISCVTargetInfo::validateAsmConstraint(
return true;
}
return false;
+ case 'P':
+ // An even-odd register pair - GPR
+ if (Name[1] == 'r') {
+ Info.setAllowsRegister();
+ Name += 1;
+ return true;
+ }
+ return false;
case 'v':
// A vector register.
if (Name[1] == 'r' || Name[1] == 'd' || Name[1] == 'm') {
@@ -122,8 +130,9 @@ bool RISCVTargetInfo::validateAsmConstraint(
std::string RISCVTargetInfo::convertConstraint(const char *&Constraint) const {
std::string R;
switch (*Constraint) {
- // c* and v* are two-letter constraints on RISC-V.
+ // c*, P*, and v* are all two-letter constraints on RISC-V.
case 'c':
+ case 'P':
case 'v':
R = std::string("^") + std::string(Constraint, 2);
Constraint += 1;
diff --git a/clang/test/CodeGen/RISCV/riscv-inline-asm.c b/clang/test/CodeGen/RISCV/riscv-inline-asm.c
index 75b91d3c497c50..eb6e42f3eb9529 100644
--- a/clang/test/CodeGen/RISCV/riscv-inline-asm.c
+++ b/clang/test/CodeGen/RISCV/riscv-inline-asm.c
@@ -33,6 +33,19 @@ void test_cf(float f, double d) {
asm volatile("" : "=cf"(cd) : "cf"(d));
}
+#if __riscv_xlen == 32
+typedef long long double_xlen_t;
+#elif __riscv_xlen == 64
+typedef __int128_t double_xlen_t;
+#endif
+double_xlen_t test_Pr_wide_scalar(double_xlen_t p) {
+// CHECK-LABEL: define{{.*}} {{i128|i64}} @test_Pr_wide_scalar(
+// CHECK: call {{i128|i64}} asm sideeffect "", "=^Pr,^Pr"({{i128|i64}} %{{.*}})
+ double_xlen_t ret;
+ asm volatile("" : "=Pr"(ret) : "Pr"(p));
+ return ret;
+}
+
void test_I(void) {
// CHECK-LABEL: define{{.*}} void @test_I()
// CHECK: call void asm sideeffect "", "I"(i32 2047)
diff --git a/llvm/include/llvm/CodeGen/ValueTypes.td b/llvm/include/llvm/CodeGen/ValueTypes.td
index 6d6b92958b4321..7a1c8624f9d258 100644
--- a/llvm/include/llvm/CodeGen/ValueTypes.td
+++ b/llvm/include/llvm/CodeGen/ValueTypes.td
@@ -317,20 +317,23 @@ def riscv_nxv16i8x3 : VTVecTup<384, 3, i8, 220>; // RISCV vector tuple(min_num_
def riscv_nxv16i8x4 : VTVecTup<512, 4, i8, 221>; // RISCV vector tuple(min_num_elts=16, nf=4)
def riscv_nxv32i8x2 : VTVecTup<512, 2, i8, 222>; // RISCV vector tuple(min_num_elts=32, nf=2)
-def x86mmx : ValueType<64, 223>; // X86 MMX value
-def Glue : ValueType<0, 224>; // Pre-RA sched glue
-def isVoid : ValueType<0, 225>; // Produces no value
-def untyped : ValueType<8, 226> { // Produces an untyped value
+def riscv_i32_pair : ValueType<64, 223>; // RISCV pair of RV32 GPRs
+def riscv_i64_pair : ValueType<128, 224>; // RISCV pair of RV64 GPRs
+
+def x86mmx : ValueType<64, 225>; // X86 MMX value
+def Glue : ValueType<0, 226>; // Pre-RA sched glue
+def isVoid : ValueType<0, 227>; // Produces no value
+def untyped : ValueType<8, 228> { // Produces an untyped value
let LLVMName = "Untyped";
}
-def funcref : ValueType<0, 227>; // WebAssembly's funcref type
-def externref : ValueType<0, 228>; // WebAssembly's externref type
-def exnref : ValueType<0, 229>; // WebAssembly's exnref type
-def x86amx : ValueType<8192, 230>; // X86 AMX value
-def i64x8 : ValueType<512, 231>; // 8 Consecutive GPRs (AArch64)
+def funcref : ValueType<0, 229>; // WebAssembly's funcref type
+def externref : ValueType<0, 230>; // WebAssembly's externref type
+def exnref : ValueType<0, 231>; // WebAssembly's exnref type
+def x86amx : ValueType<8192, 232>; // X86 AMX value
+def i64x8 : ValueType<512, 233>; // 8 Consecutive GPRs (AArch64)
def aarch64svcount
- : ValueType<16, 232>; // AArch64 predicate-as-counter
-def spirvbuiltin : ValueType<0, 233>; // SPIR-V's builtin type
+ : ValueType<16, 234>; // AArch64 predicate-as-counter
+def spirvbuiltin : ValueType<0, 235>; // SPIR-V's builtin type
let isNormalValueType = false in {
def token : ValueType<0, 504>; // TokenTy
diff --git a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
index 8287565336b54d..bf68538e083fdd 100644
--- a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
@@ -5753,7 +5753,8 @@ TargetLowering::ParseConstraints(const DataLayout &DL,
assert(!Call.getType()->isVoidTy() && "Bad inline asm!");
if (auto *STy = dyn_cast<StructType>(Call.getType())) {
OpInfo.ConstraintVT =
- getSimpleValueType(DL, STy->getElementType(ResNo));
+ getAsmOperandValueType(DL, STy->getElementType(ResNo))
+ .getSimpleVT();
} else {
assert(ResNo == 0 && "Asm only has one result!");
OpInfo.ConstraintVT =
diff --git a/llvm/lib/CodeGen/ValueTypes.cpp b/llvm/lib/CodeGen/ValueTypes.cpp
index e3c746b274dde1..7ce7102fe98a5f 100644
--- a/llvm/lib/CodeGen/ValueTypes.cpp
+++ b/llvm/lib/CodeGen/ValueTypes.cpp
@@ -177,6 +177,10 @@ std::string EVT::getEVTString() const {
if (isFloatingPoint())
return "f" + utostr(getSizeInBits());
llvm_unreachable("Invalid EVT!");
+ case MVT::riscv_i32_pair:
+ return "riscv_i32_pair";
+ case MVT::riscv_i64_pair:
+ return "riscv_i64_pair";
case MVT::bf16: return "bf16";
case MVT::ppcf128: return "ppcf128";
case MVT::isVoid: return "isVoid";
@@ -214,6 +218,8 @@ Type *EVT::getTypeForEVT(LLVMContext &Context) const {
assert(isExtended() && "Type is not extended!");
return LLVMTy;
case MVT::isVoid: return Type::getVoidTy(Context);
+ case MVT::riscv_i32_pair: return IntegerType::get(Context, 64);
+ case MVT::riscv_i64_pair: return IntegerType::get(Context, 128);
case MVT::x86mmx: return llvm::FixedVectorType::get(llvm::IntegerType::get(Context, 64), 1);
case MVT::aarch64svcount:
return TargetExtType::get(Context, "aarch64.svcount");
diff --git a/llvm/lib/Target/RISCV/AsmParser/RISCVAsmParser.cpp b/llvm/lib/Target/RISCV/AsmParser/RISCVAsmParser.cpp
index 4d46afb8c4ef97..1b23b36a59e0ec 100644
--- a/llvm/lib/Target/RISCV/AsmParser/RISCVAsmParser.cpp
+++ b/llvm/lib/Target/RISCV/AsmParser/RISCVAsmParser.cpp
@@ -481,6 +481,12 @@ struct RISCVOperand final : public MCParsedAsmOperand {
RISCVMCRegisterClasses[RISCV::GPRRegClassID].contains(Reg.RegNum);
}
+ bool isGPRPair() const {
+ return Kind == KindTy::Register &&
+ RISCVMCRegisterClasses[RISCV::GPRPairRegClassID].contains(
+ Reg.RegNum);
+ }
+
bool isGPRF16() const {
return Kind == KindTy::Register &&
RISCVMCRegisterClasses[RISCV::GPRF16RegClassID].contains(Reg.RegNum);
@@ -491,17 +497,17 @@ struct RISCVOperand final : public MCParsedAsmOperand {
RISCVMCRegisterClasses[RISCV::GPRF32RegClassID].contains(Reg.RegNum);
}
- bool isGPRAsFPR() const { return isGPR() && Reg.IsGPRAsFPR; }
- bool isGPRAsFPR16() const { return isGPRF16() && Reg.IsGPRAsFPR; }
- bool isGPRAsFPR32() const { return isGPRF32() && Reg.IsGPRAsFPR; }
- bool isGPRPairAsFPR() const { return isGPRPair() && Reg.IsGPRAsFPR; }
-
- bool isGPRPair() const {
+ bool isGPRF64Pair() const {
return Kind == KindTy::Register &&
- RISCVMCRegisterClasses[RISCV::GPRPairRegClassID].contains(
+ RISCVMCRegisterClasses[RISCV::GPRF64PairRegClassID].contains(
Reg.RegNum);
}
+ bool isGPRAsFPR() const { return isGPR() && Reg.IsGPRAsFPR; }
+ bool isGPRAsFPR16() const { return isGPRF16() && Reg.IsGPRAsFPR; }
+ bool isGPRAsFPR32() const { return isGPRF32() && Reg.IsGPRAsFPR; }
+ bool isGPRPairAsFPR64() const { return isGPRF64Pair() && Reg.IsGPRAsFPR; }
+
static bool evaluateConstantImm(const MCExpr *Expr, int64_t &Imm,
RISCVMCExpr::VariantKind &VK) {
if (auto *RE = dyn_cast<RISCVMCExpr>(Expr)) {
@@ -2399,7 +2405,7 @@ ParseStatus RISCVAsmParser::parseGPRPairAsFPR64(OperandVector &Operands) {
const MCRegisterInfo *RI = getContext().getRegisterInfo();
MCRegister Pair = RI->getMatchingSuperReg(
Reg, RISCV::sub_gpr_even,
- &RISCVMCRegisterClasses[RISCV::GPRPairRegClassID]);
+ &RISCVMCRegisterClasses[RISCV::GPRF64PairRegClassID]);
Operands.push_back(RISCVOperand::createReg(Pair, S, E, /*isGPRAsFPR=*/true));
return ParseStatus::Success;
}
diff --git a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
index 95bca928a10e95..4bc5e265e335b7 100644
--- a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
@@ -952,6 +952,35 @@ void RISCVDAGToDAGISel::Select(SDNode *Node) {
ReplaceNode(Node, Res);
return;
}
+ case RISCVISD::BuildGPRPair: {
+ SDValue Ops[] = {
+ CurDAG->getTargetConstant(RISCV::GPRPairRegClassID, DL, MVT::i32),
+ Node->getOperand(0),
+ CurDAG->getTargetConstant(RISCV::sub_gpr_even, DL, MVT::i32),
+ Node->getOperand(1),
+ CurDAG->getTargetConstant(RISCV::sub_gpr_odd, DL, MVT::i32)};
+
+ SDNode *N = CurDAG->getMachineNode(TargetOpcode::REG_SEQUENCE, DL,
+ Subtarget->getXLenPairVT(), Ops);
+ ReplaceNode(Node, N);
+ return;
+ }
+ case RISCVISD::SplitGPRPair: {
+ if (!SDValue(Node, 0).use_empty()) {
+ SDValue Lo = CurDAG->getTargetExtractSubreg(RISCV::sub_gpr_even, DL, VT,
+ Node->getOperand(0));
+ ReplaceUses(SDValue(Node, 0), Lo);
+ }
+
+ if (!SDValue(Node, 1).use_empty()) {
+ SDValue Hi = CurDAG->getTargetExtractSubreg(RISCV::sub_gpr_odd, DL, VT,
+ Node->getOperand(0));
+ ReplaceUses(SDValue(Node, 1), Hi);
+ }
+
+ CurDAG->RemoveDeadNode(Node);
+ return;
+ }
case RISCVISD::BuildPairF64: {
if (!Subtarget->hasStdExtZdinx())
break;
@@ -959,7 +988,7 @@ void RISCVDAGToDAGISel::Select(SDNode *Node) {
assert(!Subtarget->is64Bit() && "Unexpected subtarget");
SDValue Ops[] = {
- CurDAG->getTargetConstant(RISCV::GPRPairRegClassID, DL, MVT::i32),
+ CurDAG->getTargetConstant(RISCV::GPRF64PairRegClassID, DL, MVT::i32),
Node->getOperand(0),
CurDAG->getTargetConstant(RISCV::sub_gpr_even, DL, MVT::i32),
Node->getOperand(1),
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index 79c0437e9e9e9c..c785fa2b0eb168 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -113,9 +113,11 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
}
MVT XLenVT = Subtarget.getXLenVT();
+ MVT XLenPairVT = Subtarget.getXLenPairVT();
// Set up the register classes.
addRegisterClass(XLenVT, &RISCV::GPRRegClass);
+ addRegisterClass(XLenPairVT, &RISCV::GPRPairRegClass);
if (Subtarget.hasStdExtZfhmin())
addRegisterClass(MVT::f16, &RISCV::FPR16RegClass);
@@ -133,7 +135,7 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
if (Subtarget.is64Bit())
addRegisterClass(MVT::f64, &RISCV::GPRRegClass);
else
- addRegisterClass(MVT::f64, &RISCV::GPRPairRegClass);
+ addRegisterClass(MVT::f64, &RISCV::GPRF64PairRegClass);
}
static const MVT::SimpleValueType BoolVecVTs[] = {
@@ -295,6 +297,11 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
setCondCodeAction(ISD::SETLE, XLenVT, Expand);
}
+ if (Subtarget.is64Bit())
+ setOperationAction(ISD::BITCAST, MVT::i128, Custom);
+ else
+ setOperationAction(ISD::BITCAST, MVT::i64, Custom);
+
setOperationAction({ISD::STACKSAVE, ISD::STACKRESTORE}, MVT::Other, Expand);
setOperationAction(ISD::VASTART, MVT::Other, Custom);
@@ -2211,6 +2218,17 @@ bool RISCVTargetLowering::isExtractSubvectorCheap(EVT ResVT, EVT SrcVT,
return (ResElts * 2) == SrcElts && (Index == 0 || Index == ResElts);
}
+EVT RISCVTargetLowering::getAsmOperandValueType(const DataLayout &DL, Type *Ty,
+ bool AllowUnknown) const {
+ if (!Subtarget.is64Bit() && Ty->isIntegerTy(64))
+ return MVT::riscv_i32_pair;
+
+ if (Subtarget.is64Bit() && Ty->isIntegerTy(128))
+ return MVT::riscv_i64_pair;
+
+ return TargetLowering::getAsmOperandValueType(DL, Ty, AllowUnknown);
+}
+
MVT RISCVTargetLowering::getRegisterTypeForCallingConv(LLVMContext &Context,
CallingConv::ID CC,
EVT VT) const {
@@ -2225,6 +2243,17 @@ MVT RISCVTargetLowering::getRegisterTypeForCallingConv(LLVMContext &Context,
return PartVT;
}
+unsigned
+RISCVTargetLowering::getNumRegisters(LLVMContext &Context, EVT VT,
+ std::optional<MVT> RegisterVT) const {
+ // Pair inline assembly operand
+ if (VT == (Subtarget.is64Bit() ? MVT::i128 : MVT::i64) && RegisterVT &&
+ *RegisterVT == Subtarget.getXLenPairVT())
+ return 1;
+
+ return TargetLowering::getNumRegisters(Context, VT, RegisterVT);
+}
+
unsigned RISCVTargetLowering::getNumRegistersForCallingConv(LLVMContext &Context,
CallingConv::ID CC,
EVT VT) const {
@@ -2765,6 +2794,19 @@ RISCVTargetLowering::computeVLMAXBounds(MVT VecVT,
return std::make_pair(MinVLMAX, MaxVLMAX);
}
+bool RISCVTargetLowering::isLoadBitCastBeneficial(
+ EVT LoadVT, EVT BitcastVT, const SelectionDAG &DAG,
+ const MachineMemOperand &MMO) const {
+ // We want to leave `bitcasts` to/from MVT::riscv_i<xlen>_pair separate from
+ // loads/stores so they can be turned into BuildGPRPair/::SplitGPRPair nodes.
+ if (LoadVT == (Subtarget.is64Bit() ? MVT::i128 : MVT::i64) &&
+ BitcastVT == Subtarget.getXLenPairVT())
+ return false;
+
+ return TargetLoweringBase::isLoadBitCastBeneficial(LoadVT, BitcastVT, DAG,
+ MMO);
+}
+
// The state of RVV BUILD_VECTOR and VECTOR_SHUFFLE lowering is that very few
// of either is (currently) supported. This can get us into an infinite loop
// where we try to lower a BUILD_VECTOR as a VECTOR_SHUFFLE as a BUILD_VECTOR
@@ -6422,11 +6464,19 @@ SDValue RISCVTargetLowering::LowerOperation(SDValue Op,
SDValue NewOp0 = DAG.getNode(ISD::ANY_EXTEND, DL, MVT::i64, Op0);
return DAG.getNode(RISCVISD::FMV_W_X_RV64, DL, MVT::f32, NewOp0);
}
- if (VT == MVT::f64 && Op0VT == MVT::i64 && XLenVT == MVT::i32) {
+ if (VT == MVT::f64 && Op0VT == MVT::i64 && !Subtarget.is64Bit() &&
+ Subtarget.hasStdExtDOrZdinx()) {
SDValue Lo, Hi;
std::tie(Lo, Hi) = DAG.SplitScalar(Op0, DL, MVT::i32, MVT::i32);
return DAG.getNode(RISCVISD::BuildPairF64, DL, MVT::f64, Lo, Hi);
}
+ if (VT == Subtarget.getXLenPairVT() && Op0VT.isScalarInteger() &&
+ Op0VT.getSizeInBits() == 2 * Subtarget.getXLen()) {
+ SDValue Lo, Hi;
+ std::tie(Lo, Hi) = DAG.SplitScalar(Op0, DL, XLenVT, XLenVT);
+ return DAG.getNode(RISCVISD::BuildGPRPair, DL, Subtarget.getXLenPairVT(),
+ Lo, Hi);
+ }
// Consider other scalar<->scalar casts as legal if the types are legal.
// Otherwise expand them.
@@ -12942,12 +12992,21 @@ void RISCVTargetLowering::ReplaceNodeResults(SDNode *N,
SDValue FPConv =
DAG.getNode(RISCVISD::FMV_X_ANYEXTW_RV64, DL, MVT::i64, Op0);
Results.push_back(DAG.getNode(ISD::TRUNCATE, DL, MVT::i32, FPConv));
- } else if (VT == MVT::i64 && Op0VT == MVT::f64 && XLenVT == MVT::i32) {
+ } else if (VT == MVT::i64 && Op0VT == MVT::f64 && !Subtarget.is64Bit() &&
+ Subtarget.hasStdExtDOrZdinx()) {
SDValue NewReg = DAG.getNode(RISCVISD::SplitF64, DL,
DAG.getVTList(MVT::i32, MVT::i32), Op0);
SDValue RetReg = DAG.getNode(ISD::BUILD_PAIR, DL, MVT::i64,
NewReg.getValue(0), NewReg.getValue(1));
Results.push_back(RetReg);
+ } else if (VT.isInteger() &&
+ VT.getSizeInBits() == 2 * Subtarget.getXLen() &&
+ Op0VT == Subtarget.getXLenPairVT()) {
+ SDValue NewReg = DAG.getNode(RISCVISD::SplitGPRPair, DL,
+ DAG.getVTList(XLenVT, XLenVT), Op0);
+ SDValue RetReg = DAG.getNode(ISD::BUILD_PAIR, DL, VT, NewReg.getValue(0),
+ NewReg.getValue(1));
+ Results.push_back(RetReg);
} else if (!VT.isVector() && Op0VT.isFixedLengthVector() &&
isTypeLegal(Op0VT)) {
// Custom-legalize bitcasts from fixed-length vector types to illegal
@@ -20187,6 +20246,8 @@ const char *RISCVTargetLowering::getTargetNodeName(unsigned Opcode) const {
NODE_NAME_CASE(TAIL)
NODE_NAME_CASE(SELECT_CC)
NODE_NAME_CASE(BR_CC)
+ NODE_NAME_CASE(BuildGPRPair)
+ NODE_NAME_CASE(SplitGPRPair)
NODE_NAME_CASE(BuildPairF64)
NODE_NAME_CASE(SplitF64)
NODE_NAME_CASE(ADD_LO)
@@ -20463,6 +20524,8 @@ RISCVTargetLowering::getConstraintType(StringRef Constraint) const {
return C_RegisterClass;
if (Constraint == "cr" || Constraint == "cf")
return C_RegisterClass;
+ if (Constraint == "Pr")
+ return C_RegisterClass;
}
return TargetLowering::getConstraintType(Constraint);
}
@@ -20484,7 +20547,7 @@ RISCVTargetLowering::getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
if (VT == MVT::f32 && Subtarget.hasStdExtZfinx())
return std::make_pair(0U, &RISCV::GPRF32NoX0RegClass);
if (VT == MVT::f64 && Subtarget.hasStdExtZdinx() && !Subtarget.is64Bit())
- return std::make_pair(0U, &RISCV::GPRPairNoX0RegClass);
+ return std::make_pair(0U, &RISCV::GPRF64PairNoX0RegClass);
return std::make_pair(0U, &RISCV::GPRNoX0RegClass);
case 'f':
if (VT == MVT::f16) {
@@ -20501,7 +20564,7 @@ RISCVTargetLowering::getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
if (Subtarget.hasStdExtD())
return std::make_pair(0U, &RISCV::FPR64RegClass);
if (Subtarget.hasStdExtZdinx() && !Subtarget.is64Bit())
- return std::make_pair(0U, &RISCV::GPRPairNoX0RegClass);
+ return std::make_pair(0U, &RISCV::GPRF64PairNoX0RegClass);
if (Subtarget.hasStdExtZdinx() && Subtarget.is64Bit())
return std::make_pair(0U, &RISCV::GPRNoX0RegClass);
}
@@ -20543,7 +20606,7 @@ RISCVTargetLowering::getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
if (VT == MVT::f32 && Subtarget.hasStdExtZfinx())
return std::make_pair(0U, &RISCV::GPRF32CRegClass);
if (VT == MVT::f64 && Subtarget.hasStdExtZdinx() && !Subtarget.is64Bit())
- return std::make_pair(0U, &RISCV::GPRPairCRegClass);
+ return std::make_pair(0U, &RISCV::GPRF64PairCRegClass);
if (!VT.isVector())
return std::make_pair(0U, &RISCV::GPRCRegClass);
} else if (Constraint == "cf") {
@@ -20561,10 +20624,12 @@ RISCVTargetLowering::getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
if (Subtarget.hasStdExtD())
return std::make_pair(0U, &RISCV::FPR64CRegClass);
if (Subtarget.hasStdExtZdinx() && !Subtarget.is64Bit())
- return std::make_pair(0U, &RISCV::GPRPairCRegClass);
+ return std::make_pair(0U, &RISCV::GPRF64PairCRegClass);
if (Subtarget.hasStdExtZdinx() && Subtarget.is64Bit())
return std::make_pair(0U, &RISCV::GPRCRegClass);
}
+ } else if (Constraint == "Pr") {
+ return std::make_pair(0U, &RISCV::GPRPairNoX0RegClass);
}
// Clang will correctly decode the usage of register name aliases into their
@@ -20725,7 +20790,7 @@ RISCVTargetLowering::getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
// Subtarget into account.
if (Res.second == &RISCV::GPRF16RegClass ||
Res.second == &RISCV::GPRF32RegClass ||
- Res.second == &RISCV::GPRPairRegClass)
+ Res.second == &RISCV::GPRF64PairRegClass)
return std::make_pair(Res.first, &RISCV::GPRRegClass);
return Res;
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.h b/llvm/lib/Target/RISCV/RISCVISelLowering.h
index 9ae70d257fa442..4746876a41a86f 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.h
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.h
@@ -44,6 +44,18 @@ enum NodeType : unsigned {
SELECT_CC,
BR_CC,
+ /// Turn a pair of `i<xlen>`s into a `riscv_i<xlen>_pair`.
+ /// - Output: `riscv_i<xlen>_pair`
+ /// - Input 0: `i<xlen>` low-order bits, for even register.
+ /// - Input 1: `i<xlen>` high-order bits, for odd register.
+ BuildGPRPair,
+
+ /// Turn a `riscv_i<xlen>_pair` into a pair of `i<xlen>`s.
+ /// - Output 0: `i<xlen>` low-order bits, from even register.
+ /// - Output 1: `i<xlen>` high-order bits, from odd register.
+ /// - Input: `riscv_i<xlen>_pair`
+ SplitGPRPair,
+
/// Turns a pair of `i32`s into an `f64`. Needed for rv32d/ilp32.
/// - Output: `f64`.
/// - Input 0: low-order bits (31-0) (as `i32`), for even register.
@@ -542,11 +554,19 @@ class RISCVTargetLowering : public TargetLowering {
bool softPromoteHalfType() const override { return true; }
+ EVT getAsmOperandValueType(const DataLayout &DL, Type *Ty,
+ bool AllowUnknown = false) const override;
+
/// Return the register type for a given MVT, ensuring vectors are treated
/// as a series of gpr sized integers.
MVT getRegisterTypeForCallingConv(LLVMContext &Context, CallingConv::ID CC,
EVT VT) const override;
+ /// Return the number of registers for a given MVT, for inline assembly
+ unsigned
+ getNumRegisters(LLVMContext &Context, EVT VT,
+ std::optional<MVT> RegisterVT = std::nullopt) const override;
+
/// Return the number of registers for a given MVT, ensuring vectors are
/// treated as a series of gpr sized integers.
unsigned getNumRegistersForCallingConv(LLVMContext &Context,
@@ -582,6 +602,10 @@ class RISCVTargetLowering : public TargetLowering {
return false;
}
+ bool isLoadBitCastBeneficial(EVT LoadVT, EVT BitcastVT,
+ const SelectionDAG &DAG,
+ const MachineMemOperand &MMO) const override;
+
bool
shouldExpandBuildVectorWithShuffles(EVT VT,
unsigned DefinedValues) const override;
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfoD.td b/llvm/lib/Target/RISCV/RISCVInstrInfoD.td
index b5e1298c7770a8..961c22e502b522 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfoD.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfoD.td
@@ -36,7 +36,7 @@ def AddrRegImmINX : ComplexPattern<iPTR, 2, "SelectAddrRegImmRV32Zdinx">;
def GPRPairAsFPR : AsmOperandClass {
let Name = "GPRPairAsFPR";
let ParserMethod = "parseGPRPairAsFPR64";
- let PredicateMethod = "isGPRPairAsFPR";
+ let PredicateMethod = "isGPRPairAsFPR64";
let RenderMethod = "addRegOperands";
}
@@ -52,7 +52,7 @@ def FPR64INX : RegisterOperand<GPR> {
let DecoderMethod = "DecodeGPRRegisterClass";
}
-def FPR64IN32X : RegisterOperand<GPRPair> {
+def FPR64IN32X : RegisterOperand<GPRF64Pair> {
let ParserMatchClass = GPRPairAsFPR;
}
@@ -523,15 +523,15 @@ def PseudoFROUND_D_IN32X : PseudoFROUND<FPR64IN32X, f64>;
/// Loads
let isCall = 0, mayLoad = 1, mayStore = 0, Size = 8, isCodeGenOnly = 1 in
-def PseudoRV32ZdinxLD : Pseudo<(outs GPRPair:$dst), (ins GPR:$rs1, simm12:$imm12), []>;
+def PseudoRV32ZdinxLD : Pseudo<(outs GPRF64Pair:$dst), (ins GPR:$rs1, simm12:$imm12), []>;
def : Pat<(f64 (load (AddrRegImmINX (XLenVT GPR:$rs1), simm12:$imm12))),
(PseudoRV32ZdinxLD GPR:$rs1, simm12:$imm12)>;
/// Stores
let isCall = 0, mayLoad = 0, mayStore = 1, Size = 8, isCodeGenOnly = 1 in
-def PseudoRV32ZdinxSD : Pseudo<(outs), (ins GPRPair:$rs2, GPRNoX0:$rs1, simm12:$imm12), []>;
-def : Pat<(store (f64 GPRPair:$rs2), (AddrRegImmINX (XLenVT GPR:$rs1), simm12:$imm12)),
- (PseudoRV32ZdinxSD GPRPair:$rs2, GPR:$rs1, simm12:$imm12)>;
+def PseudoRV32ZdinxSD : Pseudo<(outs), (ins GPRF64Pair:$rs2, GPRNoX0:$rs1, simm12:$imm12), []>;
+def : Pat<(store (f64 GPRF64Pair:$rs2), (AddrRegImmINX (XLenVT GPR:$rs1), simm12:$imm12)),
+ (PseudoRV32ZdinxSD GPRF64Pair:$rs2, GPR:$rs1, simm12:$imm12)>;
} // Predicates = [HasStdExtZdinx, IsRV32]
let Predicates = [HasStdExtD, IsRV32] in {
diff --git a/llvm/lib/Target/RISCV/RISCVRegisterInfo.td b/llvm/lib/Target/RISCV/RISCVRegisterInfo.td
index 685f04213afa86..0acca502f10c18 100644
--- a/llvm/lib/Target/RISCV/RISCVRegisterInfo.td
+++ b/llvm/lib/Target/RISCV/RISCVRegisterInfo.td
@@ -74,7 +74,10 @@ def sub_gpr_odd : SubRegIndex<32, 32> {
}
} // Namespace = "RISCV"
-// Integer registers
+//===----------------------------------------------------------------------===//
+// General Purpose Registers (aka Integer Registers)
+//===----------------------------------------------------------------------===//
+
// CostPerUse is set higher for registers that may not be compressible as they
// are not part of GPRC, the most restrictive register class used by the
// compressed instruction set. This will influence the greedy register
@@ -210,6 +213,7 @@ def XLenFVT : ValueTypeByHwMode<[RV64],
[f64]>;
def XLenPairFVT : ValueTypeByHwMode<[RV32],
[f64]>;
+
def XLenRI : RegInfoByHwMode<
[RV32, RV64],
[RegInfo<32,32,32>, RegInfo<64,64,64>]>;
@@ -279,7 +283,67 @@ def SR07 : GPRRegisterClass<(add (sequence "X%u", 8, 9),
def GPRX1X5 : GPRRegisterClass<(add X1, X5)>;
-// Floating point registers
+//===----------------------------------------------------------------------===//
+// Even-Odd GPR Pairs
+//===----------------------------------------------------------------------===//
+
+def XLenPairVT : ValueTypeByHwMode<
+ [RV32, RV64],
+ [riscv_i32_pair, riscv_i64_pair]>;
+
+def XLenPairRI : RegInfoByHwMode<
+ [RV32, RV64],
+ [RegInfo<64, 64, 32>, RegInfo<128, 128, 64>]>;
+
+// Dummy zero register for use in the register pair containing X0 (as X1 is
+// not read to or written when the X0 register pair is used).
+def DUMMY_REG_PAIR_WITH_X0 : RISCVReg<0, "0">;
+
+// Must add DUMMY_REG_PAIR_WITH_X0 to a separate register class to prevent the
+// register's existence from changing codegen (due to the regPressureSetLimit
+// for the GPR register class being altered).
+def GPRAll : GPRRegisterClass<(add GPR, DUMMY_REG_PAIR_WITH_X0)>;
+
+let RegAltNameIndices = [ABIRegAltName] in {
+ def X0_Pair : RISCVRegWithSubRegs<0, X0.AsmName,
+ [X0, DUMMY_REG_PAIR_WITH_X0],
+ X0.AltNames> {
+ let SubRegIndices = [sub_gpr_even, sub_gpr_odd];
+ let CoveredBySubRegs = 1;
+ }
+ foreach I = 1-15 in {
+ defvar Index = !shl(I, 1);
+ defvar IndexP1 = !add(Index, 1);
+ defvar Reg = !cast<Register>("X"#Index);
+ defvar RegP1 = !cast<Register>("X"#IndexP1);
+ def "X" # Index #"_X" # IndexP1 : RISCVRegWithSubRegs<Index,
+ Reg.AsmName,
+ [Reg, RegP1],
+ Reg.AltNames> {
+ let SubRegIndices = [sub_gpr_even, sub_gpr_odd];
+ let CoveredBySubRegs = 1;
+ }
+ }
+}
+
+let RegInfos = XLenPairRI,
+ DecoderMethod = "DecodeGPRPairRegisterClass" in {
+def GPRPair : RISCVRegisterClass<[XLenPairVT], 64, (add
+ X10_X11, X12_X13, X14_X15, X16_X17,
+ X6_X7,
+ X28_X29, X30_X31,
+ X8_X9,
+ X18_X19, X20_X21, X22_X23, X24_X25, X26_X27,
+ X0_Pair, X2_X3, X4_X5
+)>;
+
+def GPRPairNoX0 : RISCVRegisterClass<[XLenPairVT], 64, (sub GPRPair, X0_Pair)>;
+} // let RegInfos = XLenPairRI, DecoderMethod = "DecodeGPRPairRegisterClass"
+
+//===----------------------------------------------------------------------===//
+// Floating Point Registers
+//===----------------------------------------------------------------------===//
+
let RegAltNameIndices = [ABIRegAltName] in {
def F0_H : RISCVReg16<0, "f0", ["ft0"]>, DwarfRegNum<[32]>;
def F1_H : RISCVReg16<1, "f1", ["ft1"]>, DwarfRegNum<[33]>;
@@ -373,8 +437,51 @@ def FPR64C : RISCVRegisterClass<[f64], 64, (add
(sequence "F%u_D", 8, 9)
)>;
+//===----------------------------------------------------------------------===//
+// GPR Classes for "H/F/D in X"
+//===----------------------------------------------------------------------===//
+
+// 16-bit GPR sub-register class used by Zhinx instructions.
+def GPRF16 : RISCVRegisterClass<[f16], 16, (add (sequence "X%u_H", 10, 17),
+ (sequence "X%u_H", 5, 7),
+ (sequence "X%u_H", 28, 31),
+ (sequence "X%u_H", 8, 9),
+ (sequence "X%u_H", 18, 27),
+ (sequence "X%u_H", 0, 4))>;
+def GPRF16C : RISCVRegisterClass<[f16], 16, (add (sequence "X%u_H", 10, 15),
+ (sequence "X%u_H", 8, 9))>;
+def GPRF16NoX0 : RISCVRegisterClass<[f16], 16, (sub GPRF16, X0_H)>;
+
+def GPRF32 : RISCVRegisterClass<[f32], 32, (add (sequence "X%u_W", 10, 17),
+ (sequence "X%u_W", 5, 7),
+ (sequence "X%u_W", 28, 31),
+ (sequence "X%u_W", 8, 9),
+ (sequence "X%u_W", 18, 27),
+ (sequence "X%u_W", 0, 4))>;
+def GPRF32C : RISCVRegisterClass<[f32], 32, (add (sequence "X%u_W", 10, 15),
+ (sequence "X%u_W", 8, 9))>;
+def GPRF32NoX0 : RISCVRegisterClass<[f32], 32, (sub GPRF32, X0_W)>;
+
+let DecoderMethod = "DecodeGPRPairRegisterClass" in
+def GPRF64Pair : RISCVRegisterClass<[XLenPairFVT], 64, (add
+ X10_X11, X12_X13, X14_X15, X16_X17,
+ X6_X7,
+ X28_X29, X30_X31,
+ X8_X9,
+ X18_X19, X20_X21, X22_X23, X24_X25, X26_X27,
+ X0_Pair, X2_X3, X4_X5
+)>;
+
+def GPRF64PairC : RISCVRegisterClass<[XLenPairFVT], 64, (add
+ X10_X11, X12_X13, X14_X15, X8_X9
+)>;
+
+def GPRF64PairNoX0 : RISCVRegisterClass<[XLenPairFVT], 64, (sub GPRF64Pair, X0_Pair)>;
+
+//===----------------------------------------------------------------------===//
// Vector type mapping to LLVM types.
-//
+//===----------------------------------------------------------------------===//
+
// The V vector extension requires that VLEN >= 128 and <= 65536.
// Additionally, the only supported ELEN values are 32 and 64,
// thus `vscale` can be defined as VLEN/64,
@@ -534,7 +641,10 @@ class VRegList<list<dag> LIn, int start, int nf, int lmul, bit isV0> {
!foreach(i, IndexSet<start, nf, lmul, isV0>.R, "v" # i));
}
+//===----------------------------------------------------------------------===//
// Vector registers
+//===----------------------------------------------------------------------===//
+
foreach Index = !range(0, 32, 1) in {
def V#Index : RISCVReg<Index, "v"#Index>, DwarfRegNum<[!add(Index, 96)]>;
}
@@ -652,80 +762,6 @@ def VRM8NoV0 : VReg<VM8VTs, (sub VRM8, V0M8), 8>;
def VMV0 : VReg<VMaskVTs, (add V0), 1>;
-// 16-bit GPR sub-register class used by Zhinx instructions.
-def GPRF16 : RISCVRegisterClass<[f16], 16, (add (sequence "X%u_H", 10, 17),
- (sequence "X%u_H", 5, 7),
- (sequence "X%u_H", 28, 31),
- (sequence "X%u_H", 8, 9),
- (sequence "X%u_H", 18, 27),
- (sequence "X%u_H", 0, 4))>;
-def GPRF16C : RISCVRegisterClass<[f16], 16, (add (sequence "X%u_H", 10, 15),
- (sequence "X%u_H", 8, 9))>;
-def GPRF16NoX0 : RISCVRegisterClass<[f16], 16, (sub GPRF16, X0_H)>;
-
-def GPRF32 : RISCVRegisterClass<[f32], 32, (add (sequence "X%u_W", 10, 17),
- (sequence "X%u_W", 5, 7),
- (sequence "X%u_W", 28, 31),
- (sequence "X%u_W", 8, 9),
- (sequence "X%u_W", 18, 27),
- (sequence "X%u_W", 0, 4))>;
-def GPRF32C : RISCVRegisterClass<[f32], 32, (add (sequence "X%u_W", 10, 15),
- (sequence "X%u_W", 8, 9))>;
-def GPRF32NoX0 : RISCVRegisterClass<[f32], 32, (sub GPRF32, X0_W)>;
-
-def XLenPairRI : RegInfoByHwMode<
- [RV32, RV64],
- [RegInfo<64, 64, 32>, RegInfo<128, 128, 64>]>;
-
-// Dummy zero register for use in the register pair containing X0 (as X1 is
-// not read to or written when the X0 register pair is used).
-def DUMMY_REG_PAIR_WITH_X0 : RISCVReg<0, "0">;
-
-// Must add DUMMY_REG_PAIR_WITH_X0 to a separate register class to prevent the
-// register's existence from changing codegen (due to the regPressureSetLimit
-// for the GPR register class being altered).
-def GPRAll : GPRRegisterClass<(add GPR, DUMMY_REG_PAIR_WITH_X0)>;
-
-let RegAltNameIndices = [ABIRegAltName] in {
- def X0_Pair : RISCVRegWithSubRegs<0, X0.AsmName,
- [X0, DUMMY_REG_PAIR_WITH_X0],
- X0.AltNames> {
- let SubRegIndices = [sub_gpr_even, sub_gpr_odd];
- let CoveredBySubRegs = 1;
- }
- foreach I = 1-15 in {
- defvar Index = !shl(I, 1);
- defvar IndexP1 = !add(Index, 1);
- defvar Reg = !cast<Register>("X"#Index);
- defvar RegP1 = !cast<Register>("X"#IndexP1);
- def "X" # Index #"_X" # IndexP1 : RISCVRegWithSubRegs<Index,
- Reg.AsmName,
- [Reg, RegP1],
- Reg.AltNames> {
- let SubRegIndices = [sub_gpr_even, sub_gpr_odd];
- let CoveredBySubRegs = 1;
- }
- }
-}
-
-let RegInfos = XLenPairRI,
- DecoderMethod = "DecodeGPRPairRegisterClass" in {
-def GPRPair : RISCVRegisterClass<[XLenPairFVT], 64, (add
- X10_X11, X12_X13, X14_X15, X16_X17,
- X6_X7,
- X28_X29, X30_X31,
- X8_X9,
- X18_X19, X20_X21, X22_X23, X24_X25, X26_X27,
- X0_Pair, X2_X3, X4_X5
-)>;
-
-def GPRPairC : RISCVRegisterClass<[XLenPairFVT], 64, (add
- X10_X11, X12_X13, X14_X15, X8_X9
-)>;
-
-def GPRPairNoX0 : RISCVRegisterClass<[XLenPairFVT], 64, (sub GPRPair, X0_Pair)>;
-} // let RegInfos = XLenPairRI, DecoderMethod = "DecodeGPRPairRegisterClass"
-
// The register class is added for inline assembly for vector mask types.
def VM : VReg<VMaskVTs, (add VR), 1>;
@@ -770,7 +806,10 @@ foreach m = LMULList in {
}
}
-// Special registers
+//===----------------------------------------------------------------------===//
+// Special Registers
+//===----------------------------------------------------------------------===//
+
def FFLAGS : RISCVReg<0, "fflags">;
def FRM : RISCVReg<0, "frm">;
diff --git a/llvm/lib/Target/RISCV/RISCVSubtarget.h b/llvm/lib/Target/RISCV/RISCVSubtarget.h
index f59a3737ae76f9..aa3f91124494bd 100644
--- a/llvm/lib/Target/RISCV/RISCVSubtarget.h
+++ b/llvm/lib/Target/RISCV/RISCVSubtarget.h
@@ -201,6 +201,10 @@ class RISCVSubtarget : public RISCVGenSubtargetInfo {
return Min;
}
+ MVT getXLenPairVT() const {
+ return is64Bit() ? MVT::riscv_i64_pair : MVT::riscv_i32_pair;
+ }
+
/// If the ElementCount or TypeSize \p X is scalable and VScale (VLEN) is
/// exactly known, returns \p X converted to a fixed quantity. Otherwise
/// returns \p X unmodified.
diff --git a/llvm/test/CodeGen/RISCV/rv32-inline-asm-pairs.ll b/llvm/test/CodeGen/RISCV/rv32-inline-asm-pairs.ll
new file mode 100644
index 00000000000000..a7f121c67e4abd
--- /dev/null
+++ b/llvm/test/CodeGen/RISCV/rv32-inline-asm-pairs.ll
@@ -0,0 +1,73 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
+; RUN: | FileCheck %s
+
+define i64 @test_Pr_wide_scalar_simple(i64 noundef %0) nounwind {
+; CHECK-LABEL: test_Pr_wide_scalar_simple:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: #APP
+; CHECK-NEXT: # a2 <- a0
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: mv a0, a2
+; CHECK-NEXT: mv a1, a3
+; CHECK-NEXT: ret
+entry:
+ %1 = call i64 asm sideeffect "/* $0 <- $1 */", "=&^Pr,^Pr"(i64 %0)
+ ret i64 %1
+}
+
+define i32 @test_Pr_wide_scalar_with_ops(i32 noundef %0) nounwind {
+; CHECK-LABEL: test_Pr_wide_scalar_with_ops:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: mv a1, a0
+; CHECK-NEXT: #APP
+; CHECK-NEXT: # a2 <- a0
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: or a0, a2, a3
+; CHECK-NEXT: ret
+entry:
+ %1 = zext i32 %0 to i64
+ %2 = shl i64 %1, 32
+ %3 = or i64 %1, %2
+ %4 = call i64 asm sideeffect "/* $0 <- $1 */", "=&^Pr,^Pr"(i64 %3)
+ %5 = trunc i64 %4 to i32
+ %6 = lshr i64 %4, 32
+ %7 = trunc i64 %6 to i32
+ %8 = or i32 %5, %7
+ ret i32 %8
+}
+
+define i64 @test_Pr_wide_scalar_inout(ptr %0, i64 noundef %1) nounwind {
+; CHECK-LABEL: test_Pr_wide_scalar_inout:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: addi sp, sp, -16
+; CHECK-NEXT: mv a3, a2
+; CHECK-NEXT: sw a0, 12(sp)
+; CHECK-NEXT: mv a2, a1
+; CHECK-NEXT: sw a1, 0(sp)
+; CHECK-NEXT: sw a3, 4(sp)
+; CHECK-NEXT: #APP
+; CHECK-NEXT: # a0; a2
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: sw a0, 12(sp)
+; CHECK-NEXT: sw a2, 0(sp)
+; CHECK-NEXT: sw a3, 4(sp)
+; CHECK-NEXT: mv a0, a2
+; CHECK-NEXT: mv a1, a3
+; CHECK-NEXT: addi sp, sp, 16
+; CHECK-NEXT: ret
+entry:
+ %2 = alloca ptr, align 4
+ %3 = alloca i64, align 8
+ store ptr %0, ptr %2, align 4
+ store i64 %1, ptr %3, align 8
+ %4 = load ptr, ptr %2, align 4
+ %5 = load i64, ptr %3, align 8
+ %6 = call { ptr, i64 } asm sideeffect "/* $0; $1 */", "=r,=^Pr,0,1"(ptr %4, i64 %5)
+ %7 = extractvalue { ptr, i64} %6, 0
+ %8 = extractvalue { ptr, i64 } %6, 1
+ store ptr %7, ptr %2, align 4
+ store i64 %8, ptr %3, align 8
+ %9 = load i64, ptr %3, align 8
+ ret i64 %9
+}
diff --git a/llvm/test/CodeGen/RISCV/rv64-inline-asm-pairs.ll b/llvm/test/CodeGen/RISCV/rv64-inline-asm-pairs.ll
new file mode 100644
index 00000000000000..d8b4b2e21c4f84
--- /dev/null
+++ b/llvm/test/CodeGen/RISCV/rv64-inline-asm-pairs.ll
@@ -0,0 +1,73 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
+; RUN: | FileCheck %s
+
+define i128 @test_Pr_wide_scalar_simple(i128 noundef %0) nounwind {
+; CHECK-LABEL: test_Pr_wide_scalar_simple:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: #APP
+; CHECK-NEXT: # a2 <- a0
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: mv a0, a2
+; CHECK-NEXT: mv a1, a3
+; CHECK-NEXT: ret
+entry:
+ %1 = call i128 asm sideeffect "/* $0 <- $1 */", "=&^Pr,^Pr"(i128 %0)
+ ret i128 %1
+}
+
+define i64 @test_Pr_wide_scalar_with_ops(i64 noundef %0) nounwind {
+; CHECK-LABEL: test_Pr_wide_scalar_with_ops:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: mv a1, a0
+; CHECK-NEXT: #APP
+; CHECK-NEXT: # a2 <- a0
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: or a0, a2, a3
+; CHECK-NEXT: ret
+entry:
+ %1 = zext i64 %0 to i128
+ %2 = shl i128 %1, 64
+ %3 = or i128 %1, %2
+ %4 = call i128 asm sideeffect "/* $0 <- $1 */", "=&^Pr,^Pr"(i128 %3)
+ %5 = trunc i128 %4 to i64
+ %6 = lshr i128 %4, 64
+ %7 = trunc i128 %6 to i64
+ %8 = or i64 %5, %7
+ ret i64 %8
+}
+
+define i128 @test_Pr_wide_scalar_inout(ptr %0, i128 noundef %1) nounwind {
+; CHECK-LABEL: test_Pr_wide_scalar_inout:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: addi sp, sp, -32
+; CHECK-NEXT: mv a3, a2
+; CHECK-NEXT: sd a0, 24(sp)
+; CHECK-NEXT: mv a2, a1
+; CHECK-NEXT: sd a1, 0(sp)
+; CHECK-NEXT: sd a3, 8(sp)
+; CHECK-NEXT: #APP
+; CHECK-NEXT: # a0; a2
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: sd a0, 24(sp)
+; CHECK-NEXT: sd a2, 0(sp)
+; CHECK-NEXT: sd a3, 8(sp)
+; CHECK-NEXT: mv a0, a2
+; CHECK-NEXT: mv a1, a3
+; CHECK-NEXT: addi sp, sp, 32
+; CHECK-NEXT: ret
+entry:
+ %2 = alloca ptr, align 8
+ %3 = alloca i128, align 16
+ store ptr %0, ptr %2, align 8
+ store i128 %1, ptr %3, align 16
+ %4 = load ptr, ptr %2, align 8
+ %5 = load i128, ptr %3, align 16
+ %6 = call { ptr, i128 } asm sideeffect "/* $0; $1 */", "=r,=^Pr,0,1"(ptr %4, i128 %5)
+ %7 = extractvalue { ptr, i128} %6, 0
+ %8 = extractvalue { ptr, i128 } %6, 1
+ store ptr %7, ptr %2, align 8
+ store i128 %8, ptr %3, align 16
+ %9 = load i128, ptr %3, align 16
+ ret i128 %9
+}
>From 7fa4c4690a30d976463df4acda1f5a60b478538b Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Thu, 24 Oct 2024 06:55:30 -0700
Subject: [PATCH 2/8] [RISCV] Allow 'Pr' for f64 on rv32 D in X
I think it's not unreasonable that people writing for just rv32 dinx
code would expect to be able to use `Pr` for double values, given that
these are actually passed in a pair of GPRs.
---
llvm/lib/Target/RISCV/RISCVISelLowering.cpp | 2 ++
.../CodeGen/RISCV/zdinx-asm-constraint.ll | 26 +++++++++++++++++++
2 files changed, 28 insertions(+)
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index c785fa2b0eb168..c566eccdd5506f 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -20629,6 +20629,8 @@ RISCVTargetLowering::getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
return std::make_pair(0U, &RISCV::GPRCRegClass);
}
} else if (Constraint == "Pr") {
+ if (VT == MVT::f64 && !Subtarget.is64Bit() && Subtarget.hasStdExtZdinx())
+ return std::make_pair(0U, &RISCV::GPRF64PairCRegClass);
return std::make_pair(0U, &RISCV::GPRPairNoX0RegClass);
}
diff --git a/llvm/test/CodeGen/RISCV/zdinx-asm-constraint.ll b/llvm/test/CodeGen/RISCV/zdinx-asm-constraint.ll
index 18bd41a210f53f..9f8acb6370c6f5 100644
--- a/llvm/test/CodeGen/RISCV/zdinx-asm-constraint.ll
+++ b/llvm/test/CodeGen/RISCV/zdinx-asm-constraint.ll
@@ -26,6 +26,32 @@ entry:
ret void
}
+define dso_local void @zdinx_asm_Pr(ptr nocapture noundef writeonly %a, double noundef %b, double noundef %c) nounwind {
+; CHECK-LABEL: zdinx_asm_Pr:
+; CHECK: # %bb.0: # %entry
+; CHECK-NEXT: addi sp, sp, -16
+; CHECK-NEXT: sw s0, 12(sp) # 4-byte Folded Spill
+; CHECK-NEXT: sw s1, 8(sp) # 4-byte Folded Spill
+; CHECK-NEXT: mv a5, a4
+; CHECK-NEXT: mv s1, a2
+; CHECK-NEXT: mv a4, a3
+; CHECK-NEXT: mv s0, a1
+; CHECK-NEXT: #APP
+; CHECK-NEXT: fsgnjx.d a2, s0, a4
+; CHECK-NEXT: #NO_APP
+; CHECK-NEXT: sw a2, 8(a0)
+; CHECK-NEXT: sw a3, 12(a0)
+; CHECK-NEXT: lw s0, 12(sp) # 4-byte Folded Reload
+; CHECK-NEXT: lw s1, 8(sp) # 4-byte Folded Reload
+; CHECK-NEXT: addi sp, sp, 16
+; CHECK-NEXT: ret
+entry:
+ %arrayidx = getelementptr inbounds double, ptr %a, i32 1
+ %0 = tail call double asm "fsgnjx.d $0, $1, $2", "=^Pr,^Pr,^Pr"(double %b, double %c)
+ store double %0, ptr %arrayidx, align 8
+ ret void
+}
+
define dso_local void @zfinx_asm(ptr nocapture noundef writeonly %a, float noundef %b, float noundef %c) nounwind {
; CHECK-LABEL: zfinx_asm:
; CHECK: # %bb.0: # %entry
>From 126e4ee22278532e0cedc38846fced6b30009e2e Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Tue, 12 Nov 2024 11:34:36 -0800
Subject: [PATCH 3/8] fixup! [RISCV] GPR Pairs for Inline Asm using `Pr`
This commit is an alternate approach that follows SystemZ in using
MVT::Untyped for GPR Pairs rather than new MVTs.
This approach does not need to use getAsmOperandValueType, so I have
also undone the fixes for this in TargetLowering.cpp. I will open a
separate PR against AArch64 to land this fix.
---
llvm/include/llvm/CodeGen/ValueTypes.td | 25 +++----
.../CodeGen/SelectionDAG/TargetLowering.cpp | 3 +-
llvm/lib/CodeGen/ValueTypes.cpp | 6 --
llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp | 2 +-
llvm/lib/Target/RISCV/RISCVISelLowering.cpp | 70 ++++++-------------
llvm/lib/Target/RISCV/RISCVISelLowering.h | 7 --
llvm/lib/Target/RISCV/RISCVRegisterInfo.td | 4 +-
llvm/lib/Target/RISCV/RISCVSubtarget.h | 4 --
8 files changed, 37 insertions(+), 84 deletions(-)
diff --git a/llvm/include/llvm/CodeGen/ValueTypes.td b/llvm/include/llvm/CodeGen/ValueTypes.td
index 7a1c8624f9d258..6d6b92958b4321 100644
--- a/llvm/include/llvm/CodeGen/ValueTypes.td
+++ b/llvm/include/llvm/CodeGen/ValueTypes.td
@@ -317,23 +317,20 @@ def riscv_nxv16i8x3 : VTVecTup<384, 3, i8, 220>; // RISCV vector tuple(min_num_
def riscv_nxv16i8x4 : VTVecTup<512, 4, i8, 221>; // RISCV vector tuple(min_num_elts=16, nf=4)
def riscv_nxv32i8x2 : VTVecTup<512, 2, i8, 222>; // RISCV vector tuple(min_num_elts=32, nf=2)
-def riscv_i32_pair : ValueType<64, 223>; // RISCV pair of RV32 GPRs
-def riscv_i64_pair : ValueType<128, 224>; // RISCV pair of RV64 GPRs
-
-def x86mmx : ValueType<64, 225>; // X86 MMX value
-def Glue : ValueType<0, 226>; // Pre-RA sched glue
-def isVoid : ValueType<0, 227>; // Produces no value
-def untyped : ValueType<8, 228> { // Produces an untyped value
+def x86mmx : ValueType<64, 223>; // X86 MMX value
+def Glue : ValueType<0, 224>; // Pre-RA sched glue
+def isVoid : ValueType<0, 225>; // Produces no value
+def untyped : ValueType<8, 226> { // Produces an untyped value
let LLVMName = "Untyped";
}
-def funcref : ValueType<0, 229>; // WebAssembly's funcref type
-def externref : ValueType<0, 230>; // WebAssembly's externref type
-def exnref : ValueType<0, 231>; // WebAssembly's exnref type
-def x86amx : ValueType<8192, 232>; // X86 AMX value
-def i64x8 : ValueType<512, 233>; // 8 Consecutive GPRs (AArch64)
+def funcref : ValueType<0, 227>; // WebAssembly's funcref type
+def externref : ValueType<0, 228>; // WebAssembly's externref type
+def exnref : ValueType<0, 229>; // WebAssembly's exnref type
+def x86amx : ValueType<8192, 230>; // X86 AMX value
+def i64x8 : ValueType<512, 231>; // 8 Consecutive GPRs (AArch64)
def aarch64svcount
- : ValueType<16, 234>; // AArch64 predicate-as-counter
-def spirvbuiltin : ValueType<0, 235>; // SPIR-V's builtin type
+ : ValueType<16, 232>; // AArch64 predicate-as-counter
+def spirvbuiltin : ValueType<0, 233>; // SPIR-V's builtin type
let isNormalValueType = false in {
def token : ValueType<0, 504>; // TokenTy
diff --git a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
index bf68538e083fdd..8287565336b54d 100644
--- a/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
@@ -5753,8 +5753,7 @@ TargetLowering::ParseConstraints(const DataLayout &DL,
assert(!Call.getType()->isVoidTy() && "Bad inline asm!");
if (auto *STy = dyn_cast<StructType>(Call.getType())) {
OpInfo.ConstraintVT =
- getAsmOperandValueType(DL, STy->getElementType(ResNo))
- .getSimpleVT();
+ getSimpleValueType(DL, STy->getElementType(ResNo));
} else {
assert(ResNo == 0 && "Asm only has one result!");
OpInfo.ConstraintVT =
diff --git a/llvm/lib/CodeGen/ValueTypes.cpp b/llvm/lib/CodeGen/ValueTypes.cpp
index 7ce7102fe98a5f..e3c746b274dde1 100644
--- a/llvm/lib/CodeGen/ValueTypes.cpp
+++ b/llvm/lib/CodeGen/ValueTypes.cpp
@@ -177,10 +177,6 @@ std::string EVT::getEVTString() const {
if (isFloatingPoint())
return "f" + utostr(getSizeInBits());
llvm_unreachable("Invalid EVT!");
- case MVT::riscv_i32_pair:
- return "riscv_i32_pair";
- case MVT::riscv_i64_pair:
- return "riscv_i64_pair";
case MVT::bf16: return "bf16";
case MVT::ppcf128: return "ppcf128";
case MVT::isVoid: return "isVoid";
@@ -218,8 +214,6 @@ Type *EVT::getTypeForEVT(LLVMContext &Context) const {
assert(isExtended() && "Type is not extended!");
return LLVMTy;
case MVT::isVoid: return Type::getVoidTy(Context);
- case MVT::riscv_i32_pair: return IntegerType::get(Context, 64);
- case MVT::riscv_i64_pair: return IntegerType::get(Context, 128);
case MVT::x86mmx: return llvm::FixedVectorType::get(llvm::IntegerType::get(Context, 64), 1);
case MVT::aarch64svcount:
return TargetExtType::get(Context, "aarch64.svcount");
diff --git a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
index 4bc5e265e335b7..39528a8511b141 100644
--- a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
@@ -961,7 +961,7 @@ void RISCVDAGToDAGISel::Select(SDNode *Node) {
CurDAG->getTargetConstant(RISCV::sub_gpr_odd, DL, MVT::i32)};
SDNode *N = CurDAG->getMachineNode(TargetOpcode::REG_SEQUENCE, DL,
- Subtarget->getXLenPairVT(), Ops);
+ MVT::Untyped, Ops);
ReplaceNode(Node, N);
return;
}
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index c566eccdd5506f..8475a67d05e3cb 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -113,11 +113,9 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
}
MVT XLenVT = Subtarget.getXLenVT();
- MVT XLenPairVT = Subtarget.getXLenPairVT();
// Set up the register classes.
addRegisterClass(XLenVT, &RISCV::GPRRegClass);
- addRegisterClass(XLenPairVT, &RISCV::GPRPairRegClass);
if (Subtarget.hasStdExtZfhmin())
addRegisterClass(MVT::f16, &RISCV::FPR16RegClass);
@@ -297,11 +295,6 @@ RISCVTargetLowering::RISCVTargetLowering(const TargetMachine &TM,
setCondCodeAction(ISD::SETLE, XLenVT, Expand);
}
- if (Subtarget.is64Bit())
- setOperationAction(ISD::BITCAST, MVT::i128, Custom);
- else
- setOperationAction(ISD::BITCAST, MVT::i64, Custom);
-
setOperationAction({ISD::STACKSAVE, ISD::STACKRESTORE}, MVT::Other, Expand);
setOperationAction(ISD::VASTART, MVT::Other, Custom);
@@ -2218,17 +2211,6 @@ bool RISCVTargetLowering::isExtractSubvectorCheap(EVT ResVT, EVT SrcVT,
return (ResElts * 2) == SrcElts && (Index == 0 || Index == ResElts);
}
-EVT RISCVTargetLowering::getAsmOperandValueType(const DataLayout &DL, Type *Ty,
- bool AllowUnknown) const {
- if (!Subtarget.is64Bit() && Ty->isIntegerTy(64))
- return MVT::riscv_i32_pair;
-
- if (Subtarget.is64Bit() && Ty->isIntegerTy(128))
- return MVT::riscv_i64_pair;
-
- return TargetLowering::getAsmOperandValueType(DL, Ty, AllowUnknown);
-}
-
MVT RISCVTargetLowering::getRegisterTypeForCallingConv(LLVMContext &Context,
CallingConv::ID CC,
EVT VT) const {
@@ -2248,7 +2230,7 @@ RISCVTargetLowering::getNumRegisters(LLVMContext &Context, EVT VT,
std::optional<MVT> RegisterVT) const {
// Pair inline assembly operand
if (VT == (Subtarget.is64Bit() ? MVT::i128 : MVT::i64) && RegisterVT &&
- *RegisterVT == Subtarget.getXLenPairVT())
+ *RegisterVT == MVT::Untyped)
return 1;
return TargetLowering::getNumRegisters(Context, VT, RegisterVT);
@@ -2794,19 +2776,6 @@ RISCVTargetLowering::computeVLMAXBounds(MVT VecVT,
return std::make_pair(MinVLMAX, MaxVLMAX);
}
-bool RISCVTargetLowering::isLoadBitCastBeneficial(
- EVT LoadVT, EVT BitcastVT, const SelectionDAG &DAG,
- const MachineMemOperand &MMO) const {
- // We want to leave `bitcasts` to/from MVT::riscv_i<xlen>_pair separate from
- // loads/stores so they can be turned into BuildGPRPair/::SplitGPRPair nodes.
- if (LoadVT == (Subtarget.is64Bit() ? MVT::i128 : MVT::i64) &&
- BitcastVT == Subtarget.getXLenPairVT())
- return false;
-
- return TargetLoweringBase::isLoadBitCastBeneficial(LoadVT, BitcastVT, DAG,
- MMO);
-}
-
// The state of RVV BUILD_VECTOR and VECTOR_SHUFFLE lowering is that very few
// of either is (currently) supported. This can get us into an infinite loop
// where we try to lower a BUILD_VECTOR as a VECTOR_SHUFFLE as a BUILD_VECTOR
@@ -6470,13 +6439,6 @@ SDValue RISCVTargetLowering::LowerOperation(SDValue Op,
std::tie(Lo, Hi) = DAG.SplitScalar(Op0, DL, MVT::i32, MVT::i32);
return DAG.getNode(RISCVISD::BuildPairF64, DL, MVT::f64, Lo, Hi);
}
- if (VT == Subtarget.getXLenPairVT() && Op0VT.isScalarInteger() &&
- Op0VT.getSizeInBits() == 2 * Subtarget.getXLen()) {
- SDValue Lo, Hi;
- std::tie(Lo, Hi) = DAG.SplitScalar(Op0, DL, XLenVT, XLenVT);
- return DAG.getNode(RISCVISD::BuildGPRPair, DL, Subtarget.getXLenPairVT(),
- Lo, Hi);
- }
// Consider other scalar<->scalar casts as legal if the types are legal.
// Otherwise expand them.
@@ -12999,14 +12961,6 @@ void RISCVTargetLowering::ReplaceNodeResults(SDNode *N,
SDValue RetReg = DAG.getNode(ISD::BUILD_PAIR, DL, MVT::i64,
NewReg.getValue(0), NewReg.getValue(1));
Results.push_back(RetReg);
- } else if (VT.isInteger() &&
- VT.getSizeInBits() == 2 * Subtarget.getXLen() &&
- Op0VT == Subtarget.getXLenPairVT()) {
- SDValue NewReg = DAG.getNode(RISCVISD::SplitGPRPair, DL,
- DAG.getVTList(XLenVT, XLenVT), Op0);
- SDValue RetReg = DAG.getNode(ISD::BUILD_PAIR, DL, VT, NewReg.getValue(0),
- NewReg.getValue(1));
- Results.push_back(RetReg);
} else if (!VT.isVector() && Op0VT.isFixedLengthVector() &&
isTypeLegal(Op0VT)) {
// Custom-legalize bitcasts from fixed-length vector types to illegal
@@ -21418,6 +21372,17 @@ bool RISCVTargetLowering::splitValueIntoRegisterParts(
unsigned NumParts, MVT PartVT, std::optional<CallingConv::ID> CC) const {
bool IsABIRegCopy = CC.has_value();
EVT ValueVT = Val.getValueType();
+
+ if (ValueVT == (Subtarget.is64Bit() ? MVT::i128 : MVT::i64) &&
+ NumParts == 1 && PartVT == MVT::Untyped) {
+ // Pairs in Inline Assembly
+ MVT XLenVT = Subtarget.getXLenVT();
+ SDValue Lo, Hi;
+ std::tie(Lo, Hi) = DAG.SplitScalar(Val, DL, XLenVT, XLenVT);
+ Parts[0] = DAG.getNode(RISCVISD::BuildGPRPair, DL, MVT::Untyped, Lo, Hi);
+ return true;
+ }
+
if (IsABIRegCopy && (ValueVT == MVT::f16 || ValueVT == MVT::bf16) &&
PartVT == MVT::f32) {
// Cast the [b]f16 to i16, extend to i32, pad with ones to make a float
@@ -21489,6 +21454,17 @@ SDValue RISCVTargetLowering::joinRegisterPartsIntoValue(
SelectionDAG &DAG, const SDLoc &DL, const SDValue *Parts, unsigned NumParts,
MVT PartVT, EVT ValueVT, std::optional<CallingConv::ID> CC) const {
bool IsABIRegCopy = CC.has_value();
+
+ if (ValueVT == (Subtarget.is64Bit() ? MVT::i128 : MVT::i64) &&
+ NumParts == 1 && PartVT == MVT::Untyped) {
+ // Pairs in Inline Assembly
+ MVT XLenVT = Subtarget.getXLenVT();
+ SDValue Res = DAG.getNode(RISCVISD::SplitGPRPair, DL,
+ DAG.getVTList(XLenVT, XLenVT), Parts[0]);
+ return DAG.getNode(ISD::BUILD_PAIR, DL, ValueVT, Res.getValue(0),
+ Res.getValue(1));
+ }
+
if (IsABIRegCopy && (ValueVT == MVT::f16 || ValueVT == MVT::bf16) &&
PartVT == MVT::f32) {
SDValue Val = Parts[0];
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.h b/llvm/lib/Target/RISCV/RISCVISelLowering.h
index 4746876a41a86f..828bb833ecc3b7 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.h
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.h
@@ -554,9 +554,6 @@ class RISCVTargetLowering : public TargetLowering {
bool softPromoteHalfType() const override { return true; }
- EVT getAsmOperandValueType(const DataLayout &DL, Type *Ty,
- bool AllowUnknown = false) const override;
-
/// Return the register type for a given MVT, ensuring vectors are treated
/// as a series of gpr sized integers.
MVT getRegisterTypeForCallingConv(LLVMContext &Context, CallingConv::ID CC,
@@ -602,10 +599,6 @@ class RISCVTargetLowering : public TargetLowering {
return false;
}
- bool isLoadBitCastBeneficial(EVT LoadVT, EVT BitcastVT,
- const SelectionDAG &DAG,
- const MachineMemOperand &MMO) const override;
-
bool
shouldExpandBuildVectorWithShuffles(EVT VT,
unsigned DefinedValues) const override;
diff --git a/llvm/lib/Target/RISCV/RISCVRegisterInfo.td b/llvm/lib/Target/RISCV/RISCVRegisterInfo.td
index 0acca502f10c18..b2dc9407e0bba9 100644
--- a/llvm/lib/Target/RISCV/RISCVRegisterInfo.td
+++ b/llvm/lib/Target/RISCV/RISCVRegisterInfo.td
@@ -287,9 +287,7 @@ def GPRX1X5 : GPRRegisterClass<(add X1, X5)>;
// Even-Odd GPR Pairs
//===----------------------------------------------------------------------===//
-def XLenPairVT : ValueTypeByHwMode<
- [RV32, RV64],
- [riscv_i32_pair, riscv_i64_pair]>;
+defvar XLenPairVT = untyped;
def XLenPairRI : RegInfoByHwMode<
[RV32, RV64],
diff --git a/llvm/lib/Target/RISCV/RISCVSubtarget.h b/llvm/lib/Target/RISCV/RISCVSubtarget.h
index aa3f91124494bd..f59a3737ae76f9 100644
--- a/llvm/lib/Target/RISCV/RISCVSubtarget.h
+++ b/llvm/lib/Target/RISCV/RISCVSubtarget.h
@@ -201,10 +201,6 @@ class RISCVSubtarget : public RISCVGenSubtargetInfo {
return Min;
}
- MVT getXLenPairVT() const {
- return is64Bit() ? MVT::riscv_i64_pair : MVT::riscv_i32_pair;
- }
-
/// If the ElementCount or TypeSize \p X is scalable and VScale (VLEN) is
/// exactly known, returns \p X converted to a fixed quantity. Otherwise
/// returns \p X unmodified.
>From 8956f721060a76d019c3401c523806c03b7e2679 Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Tue, 12 Nov 2024 11:44:26 -0800
Subject: [PATCH 4/8] fixup! fixup! [RISCV] GPR Pairs for Inline Asm using `Pr`
---
llvm/lib/Target/RISCV/RISCVISelLowering.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.h b/llvm/lib/Target/RISCV/RISCVISelLowering.h
index 828bb833ecc3b7..773729d69a143f 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.h
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.h
@@ -44,16 +44,16 @@ enum NodeType : unsigned {
SELECT_CC,
BR_CC,
- /// Turn a pair of `i<xlen>`s into a `riscv_i<xlen>_pair`.
- /// - Output: `riscv_i<xlen>_pair`
+ /// Turn a pair of `i<xlen>`s into an even-odd register pair (`untyped`).
+ /// - Output: `untyped` even-odd register pair
/// - Input 0: `i<xlen>` low-order bits, for even register.
/// - Input 1: `i<xlen>` high-order bits, for odd register.
BuildGPRPair,
- /// Turn a `riscv_i<xlen>_pair` into a pair of `i<xlen>`s.
+ /// Turn an even-odd register pair (`untyped`) into a pair of `i<xlen>`s.
/// - Output 0: `i<xlen>` low-order bits, from even register.
/// - Output 1: `i<xlen>` high-order bits, from odd register.
- /// - Input: `riscv_i<xlen>_pair`
+ /// - Input: `untyped` even-odd register pair
SplitGPRPair,
/// Turns a pair of `i32`s into an `f64`. Needed for rv32d/ilp32.
>From 555d23026b9e7377a34ecd18a82c0b7255c7c096 Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Tue, 12 Nov 2024 11:46:53 -0800
Subject: [PATCH 5/8] fixup! fixup! fixup! [RISCV] GPR Pairs for Inline Asm
using `Pr`
---
llvm/lib/Target/RISCV/RISCVISelLowering.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index 8475a67d05e3cb..7a62cfcff19b86 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -21462,7 +21462,7 @@ SDValue RISCVTargetLowering::joinRegisterPartsIntoValue(
SDValue Res = DAG.getNode(RISCVISD::SplitGPRPair, DL,
DAG.getVTList(XLenVT, XLenVT), Parts[0]);
return DAG.getNode(ISD::BUILD_PAIR, DL, ValueVT, Res.getValue(0),
- Res.getValue(1));
+ Res.getValue(1));
}
if (IsABIRegCopy && (ValueVT == MVT::f16 || ValueVT == MVT::bf16) &&
>From 752b28ecbf000d0cf4dbbcbeb8a6b5bda4cecaa5 Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Wed, 13 Nov 2024 05:03:09 -0800
Subject: [PATCH 6/8] fixup! [RISCV] GPR Pairs for Inline Asm using `Pr`
---
llvm/lib/Target/RISCV/RISCVISelLowering.cpp | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index 7a62cfcff19b86..6794ef26952045 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -21377,8 +21377,7 @@ bool RISCVTargetLowering::splitValueIntoRegisterParts(
NumParts == 1 && PartVT == MVT::Untyped) {
// Pairs in Inline Assembly
MVT XLenVT = Subtarget.getXLenVT();
- SDValue Lo, Hi;
- std::tie(Lo, Hi) = DAG.SplitScalar(Val, DL, XLenVT, XLenVT);
+ auto [Lo, Hi] = DAG.SplitScalar(Val, DL, XLenVT, XLenVT);
Parts[0] = DAG.getNode(RISCVISD::BuildGPRPair, DL, MVT::Untyped, Lo, Hi);
return true;
}
>From 22478c932d75cf37e7ecdea5f3bc53b6c0404259 Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Wed, 13 Nov 2024 06:04:26 -0800
Subject: [PATCH 7/8] fixup! [RISCV] GPR Pairs for Inline Asm using `Pr`
---
llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp | 52 +++++++--------------
1 file changed, 17 insertions(+), 35 deletions(-)
diff --git a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
index 39528a8511b141..0d7dfb84156eca 100644
--- a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
@@ -952,56 +952,36 @@ void RISCVDAGToDAGISel::Select(SDNode *Node) {
ReplaceNode(Node, Res);
return;
}
- case RISCVISD::BuildGPRPair: {
- SDValue Ops[] = {
- CurDAG->getTargetConstant(RISCV::GPRPairRegClassID, DL, MVT::i32),
- Node->getOperand(0),
- CurDAG->getTargetConstant(RISCV::sub_gpr_even, DL, MVT::i32),
- Node->getOperand(1),
- CurDAG->getTargetConstant(RISCV::sub_gpr_odd, DL, MVT::i32)};
-
- SDNode *N = CurDAG->getMachineNode(TargetOpcode::REG_SEQUENCE, DL,
- MVT::Untyped, Ops);
- ReplaceNode(Node, N);
- return;
- }
- case RISCVISD::SplitGPRPair: {
- if (!SDValue(Node, 0).use_empty()) {
- SDValue Lo = CurDAG->getTargetExtractSubreg(RISCV::sub_gpr_even, DL, VT,
- Node->getOperand(0));
- ReplaceUses(SDValue(Node, 0), Lo);
- }
-
- if (!SDValue(Node, 1).use_empty()) {
- SDValue Hi = CurDAG->getTargetExtractSubreg(RISCV::sub_gpr_odd, DL, VT,
- Node->getOperand(0));
- ReplaceUses(SDValue(Node, 1), Hi);
- }
-
- CurDAG->RemoveDeadNode(Node);
- return;
- }
+ case RISCVISD::BuildGPRPair:
case RISCVISD::BuildPairF64: {
- if (!Subtarget->hasStdExtZdinx())
+ if (Opcode == RISCVISD::BuildPairF64 && !Subtarget->hasStdExtZdinx())
break;
- assert(!Subtarget->is64Bit() && "Unexpected subtarget");
+ assert((!Subtarget->is64Bit() || Opcode == RISCVISD::BuildGPRPair) &&
+ "BuildPairF64 only handled here on rv32i_zdinx");
+
+ int RegClassID = (Opcode == RISCVISD::BuildGPRPair)
+ ? RISCV::GPRPairRegClassID
+ : RISCV::GPRF64PairRegClassID;
+ MVT OutType = (Opcode == RISCVISD::BuildGPRPair) ? MVT::Untyped : MVT::f64;
SDValue Ops[] = {
- CurDAG->getTargetConstant(RISCV::GPRF64PairRegClassID, DL, MVT::i32),
+ CurDAG->getTargetConstant(RegClassID, DL, MVT::i32),
Node->getOperand(0),
CurDAG->getTargetConstant(RISCV::sub_gpr_even, DL, MVT::i32),
Node->getOperand(1),
CurDAG->getTargetConstant(RISCV::sub_gpr_odd, DL, MVT::i32)};
SDNode *N =
- CurDAG->getMachineNode(TargetOpcode::REG_SEQUENCE, DL, MVT::f64, Ops);
+ CurDAG->getMachineNode(TargetOpcode::REG_SEQUENCE, DL, OutType, Ops);
ReplaceNode(Node, N);
return;
}
+ case RISCVISD::SplitGPRPair:
case RISCVISD::SplitF64: {
- if (Subtarget->hasStdExtZdinx()) {
- assert(!Subtarget->is64Bit() && "Unexpected subtarget");
+ if (Subtarget->hasStdExtZdinx() || Opcode != RISCVISD::SplitF64) {
+ assert((!Subtarget->is64Bit() || Opcode == RISCVISD::SplitGPRPair) &&
+ "SplitF64 only handled here on rv32i_zdinx");
if (!SDValue(Node, 0).use_empty()) {
SDValue Lo = CurDAG->getTargetExtractSubreg(RISCV::sub_gpr_even, DL, VT,
@@ -1019,6 +999,8 @@ void RISCVDAGToDAGISel::Select(SDNode *Node) {
return;
}
+ assert(Opcode != RISCVISD::SplitGPRPair && "SplitGPRPair should already be handled");
+
if (!Subtarget->hasStdExtZfa())
break;
assert(Subtarget->hasStdExtD() && !Subtarget->is64Bit() &&
>From 8d538f83a40c993c1af4789b958878fed388c035 Mon Sep 17 00:00:00 2001
From: Sam Elliott <quic_aelliott at quicinc.com>
Date: Wed, 13 Nov 2024 06:05:35 -0800
Subject: [PATCH 8/8] fixup! [RISCV] GPR Pairs for Inline Asm using `Pr`
---
llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
index 0d7dfb84156eca..472a68360c8ba2 100644
--- a/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp
@@ -999,7 +999,8 @@ void RISCVDAGToDAGISel::Select(SDNode *Node) {
return;
}
- assert(Opcode != RISCVISD::SplitGPRPair && "SplitGPRPair should already be handled");
+ assert(Opcode != RISCVISD::SplitGPRPair &&
+ "SplitGPRPair should already be handled");
if (!Subtarget->hasStdExtZfa())
break;
More information about the cfe-commits
mailing list