[clang] [llvm] [BPF] Add UHMUL and SHMUL instruction variants with -mcpu=v5 (PR #234076)
Yazhou Tang via cfe-commits
cfe-commits at lists.llvm.org
Tue Oct 20 21:58:20 PDT 2026
https://github.com/ADSWT518 created https://github.com/llvm/llvm-project/pull/234076
This patch adds LLVM support for the unsigned and signed high-half multiply instructions (UHMUL and SHMUL) proposed for the BPF ISA [1].
The instructions use the existing `BPF_MUL` opcode with the offset field distinguishing the variants:
- `off = 1`: UHMUL, returning the upper 64 bits of an unsigned 64×64-bit product.
- `off = 2`: SHMUL, returning the upper 64 bits of a signed 64×64-bit product.
Both register and immediate forms are supported. For example:
```asm
r1 uh*= r2
r3 sh*= r4
r5 uh*= -2
r6 sh*= -3
```
The backend selects these instructions for 64-bit `ISD::MULHU` and `ISD::MULHS` operations.
### CPU v5
As suggested by Alexei Starovoitov [2], this patch introduces `-mcpu=v5` instead of extending `-mcpu=v4`.
Since UHMUL and SHMUL are new ISA extensions, enabling them under v4 would change the instruction set available to existing v4 targets and could generate instructions unsupported by those targets.
CPU v5 inherits all v4 features and additionally enables UHMUL and SHMUL. The existing v4 code generation remains unchanged.
Clang also defines `__BPF_FEATURE_HMUL` for v5, and the `-disable-hmul` option allows the backend to disable generation of these instructions.
The patch includes assembly/disassembly tests, code generation tests, and Clang CPU feature tests. It also verifies that earlier CPU versions and v5 with `-disable-hmul` retain the existing lowering behavior.
The corresponding Linux kernel changes are still under review.
[1] Kernel patch series v1: https://lore.kernel.org/bpf/20260917041712.1860722-1-tangyazhou@zju.edu.cn/
[2] https://lore.kernel.org/bpf/DLHSD07GRG0W.16CFRV44ZE1DD@gmail.com/
>From 340931a49dc70032fc540ba550153c3e37273170 Mon Sep 17 00:00:00 2001
From: Tianci Cao <ziye at zju.edu.cn>
Date: Sun, 12 Jul 2026 00:51:54 +0800
Subject: [PATCH 1/3] [BPF] Add UHMUL and SHMUL instruction variants
Add assembly, disassembly, and instruction selection support for the
ALU64 high-half multiply variants encoded in the offset field of BPF_MUL:
uh*= unsigned high-half multiply (UHMUL, off=1)
sh*= signed high-half multiply (SHMUL, off=2)
Match ISD::MULHU and ISD::MULHS to the new instructions for cpu v4.
Keep ALU32, earlier CPUs, and configurations using -disable-hmul on the
existing expansion path.
Define __BPF_FEATURE_HMUL for cpu v4 so source code can detect compiler
support for the new instructions.
Co-authored-by: Yazhou Tang <tangyazhou518 at outlook.com>
Co-authored-by: Shenghao Yuan <shenghaoyuan0928 at 163.com>
---
clang/lib/Basic/Targets/BPF.cpp | 1 +
.../test/Preprocessor/bpf-predefined-macros.c | 5 ++
.../lib/Target/BPF/AsmParser/BPFAsmParser.cpp | 2 +
llvm/lib/Target/BPF/BPFISelLowering.cpp | 6 +-
llvm/lib/Target/BPF/BPFInstrInfo.td | 15 +++-
llvm/lib/Target/BPF/BPFSubtarget.cpp | 5 ++
llvm/lib/Target/BPF/BPFSubtarget.h | 3 +-
.../CodeGen/BPF/assembler-disassembler-v4.s | 9 ++
llvm/test/CodeGen/BPF/hmul.ll | 83 +++++++++++++++++++
llvm/test/MC/BPF/bad-hmul.s | 5 ++
10 files changed, 130 insertions(+), 4 deletions(-)
create mode 100644 llvm/test/CodeGen/BPF/hmul.ll
create mode 100644 llvm/test/MC/BPF/bad-hmul.s
diff --git a/clang/lib/Basic/Targets/BPF.cpp b/clang/lib/Basic/Targets/BPF.cpp
index 29e4c86ad88621..6efdd023b32d14 100644
--- a/clang/lib/Basic/Targets/BPF.cpp
+++ b/clang/lib/Basic/Targets/BPF.cpp
@@ -78,6 +78,7 @@ void BPFTargetInfo::getTargetDefines(const LangOptions &Opts,
Builder.defineMacro("__BPF_FEATURE_ST");
Builder.defineMacro("__BPF_FEATURE_LOAD_ACQ_STORE_REL");
Builder.defineMacro("__BPF_FEATURE_GOTOX");
+ Builder.defineMacro("__BPF_FEATURE_HMUL");
}
}
diff --git a/clang/test/Preprocessor/bpf-predefined-macros.c b/clang/test/Preprocessor/bpf-predefined-macros.c
index d3d59a979e06bf..c81bfa871a440d 100644
--- a/clang/test/Preprocessor/bpf-predefined-macros.c
+++ b/clang/test/Preprocessor/bpf-predefined-macros.c
@@ -82,6 +82,9 @@ int y;
#ifdef __BPF_FEATURE_UNWIND
int z;
#endif
+#ifdef __BPF_FEATURE_HMUL
+int aa;
+#endif
// CHECK: int b;
// CHECK: int c;
@@ -139,6 +142,8 @@ int z;
// CPU_V3: int z;
// CPU_V4: int z;
+// CPU_V4: int aa;
+
// CPU_GENERIC: int g;
// CPU_PROBE: int f;
diff --git a/llvm/lib/Target/BPF/AsmParser/BPFAsmParser.cpp b/llvm/lib/Target/BPF/AsmParser/BPFAsmParser.cpp
index ec167a680cdc1b..6483d02aa6f87b 100644
--- a/llvm/lib/Target/BPF/AsmParser/BPFAsmParser.cpp
+++ b/llvm/lib/Target/BPF/AsmParser/BPFAsmParser.cpp
@@ -268,6 +268,8 @@ struct BPFOperand : public MCParsedAsmOperand {
.Case("ll", true)
.Case("skb", true)
.Case("s", true)
+ .Case("uh", true)
+ .Case("sh", true)
.Case("atomic_fetch_add", true)
.Case("atomic_fetch_and", true)
.Case("atomic_fetch_or", true)
diff --git a/llvm/lib/Target/BPF/BPFISelLowering.cpp b/llvm/lib/Target/BPF/BPFISelLowering.cpp
index 435f0e15a25c59..204905a3a1c526 100644
--- a/llvm/lib/Target/BPF/BPFISelLowering.cpp
+++ b/llvm/lib/Target/BPF/BPFISelLowering.cpp
@@ -127,8 +127,10 @@ BPFTargetLowering::BPFTargetLowering(const TargetMachine &TM,
setOperationAction(ISD::SDIV, VT, Custom);
setOperationAction(ISD::SREM, VT, Custom);
}
- setOperationAction(ISD::MULHU, VT, Expand);
- setOperationAction(ISD::MULHS, VT, Expand);
+ if (VT == MVT::i32 || !STI.hasHmul()) {
+ setOperationAction(ISD::MULHU, VT, Expand);
+ setOperationAction(ISD::MULHS, VT, Expand);
+ }
setOperationAction(ISD::UMUL_LOHI, VT, Expand);
setOperationAction(ISD::SMUL_LOHI, VT, Expand);
setOperationAction(ISD::ROTR, VT, Expand);
diff --git a/llvm/lib/Target/BPF/BPFInstrInfo.td b/llvm/lib/Target/BPF/BPFInstrInfo.td
index 8c3e09009a5e50..0e5d7dce3e4da3 100644
--- a/llvm/lib/Target/BPF/BPFInstrInfo.td
+++ b/llvm/lib/Target/BPF/BPFInstrInfo.td
@@ -71,6 +71,7 @@ def BPFHasLdsx : Predicate<"Subtarget->hasLdsx()">;
def BPFHasMovsx : Predicate<"Subtarget->hasMovsx()">;
def BPFHasBswap : Predicate<"Subtarget->hasBswap()">;
def BPFHasSdivSmod : Predicate<"Subtarget->hasSdivSmod()">;
+def BPFHasHmul : Predicate<"Subtarget->hasHmul()">;
def BPFNoMovsx : Predicate<"!Subtarget->hasMovsx()">;
def BPFNoBswap : Predicate<"!Subtarget->hasBswap()">;
def BPFHasStoreImm : Predicate<"Subtarget->hasStoreImm()">;
@@ -360,7 +361,7 @@ class ALU_RR<BPFOpClass Class, BPFArithOp Opc, int off,
let BPFClass = Class;
}
-multiclass ALU<BPFArithOp Opc, int off, string OpcodeStr, SDNode OpNode> {
+multiclass ALU64<BPFArithOp Opc, int off, string OpcodeStr, SDNode OpNode> {
def _rr : ALU_RR<BPF_ALU64, Opc, off,
(outs GPR:$dst),
(ins GPR:$src2, GPR:$src),
@@ -371,6 +372,10 @@ multiclass ALU<BPFArithOp Opc, int off, string OpcodeStr, SDNode OpNode> {
(ins GPR:$src2, i64imm:$imm),
"$dst "#OpcodeStr#" $imm",
[(set GPR:$dst, (OpNode GPR:$src2, i64immSExt32:$imm))]>;
+}
+
+multiclass ALU<BPFArithOp Opc, int off, string OpcodeStr, SDNode OpNode>
+ : ALU64<Opc, off, OpcodeStr, OpNode> {
def _rr_32 : ALU_RR<BPF_ALU, Opc, off,
(outs GPR32:$dst),
(ins GPR32:$src2, GPR32:$src),
@@ -402,6 +407,14 @@ let isAsCheapAsAMove = 1 in {
defm SDIV : ALU<BPF_DIV, 1, "s/=", sdiv>;
defm SMOD : ALU<BPF_MOD, 1, "s%=", srem>;
}
+
+ // High-half multiply variants encoded via insn->off on BPF_MUL:
+ // off == 1 UHMUL: dst = ((u128)dst * (u128)src) >> 64
+ // off == 2 SHMUL: dst = ((i128)dst * (i128)src) >> 64
+ let Predicates = [BPFHasHmul] in {
+ defm UHMUL : ALU64<BPF_MUL, 1, "uh*=", mulhu>;
+ defm SHMUL : ALU64<BPF_MUL, 2, "sh*=", mulhs>;
+ }
}
class NEG_RR<BPFOpClass Class, BPFArithOp Opc,
diff --git a/llvm/lib/Target/BPF/BPFSubtarget.cpp b/llvm/lib/Target/BPF/BPFSubtarget.cpp
index 726f8f4b39827f..4e9a19a235300a 100644
--- a/llvm/lib/Target/BPF/BPFSubtarget.cpp
+++ b/llvm/lib/Target/BPF/BPFSubtarget.cpp
@@ -45,6 +45,9 @@ static cl::opt<bool> Disable_load_acq_store_rel(
cl::desc("Disable load-acquire and store-release insns"));
static cl::opt<bool> Disable_gotox("disable-gotox", cl::Hidden, cl::init(false),
cl::desc("Disable gotox insn"));
+static cl::opt<bool> Disable_hmul(
+ "disable-hmul", cl::Hidden, cl::init(false),
+ cl::desc("Disable high-half multiply (UHMUL/SHMUL) insns"));
void BPFSubtarget::anchor() {}
@@ -69,6 +72,7 @@ void BPFSubtarget::initializeEnvironment() {
HasStoreImm = false;
HasLoadAcqStoreRel = false;
HasGotox = false;
+ HasHmul = false;
AllowsMisalignedMemAccess = false;
}
@@ -101,6 +105,7 @@ void BPFSubtarget::initSubtargetFeatures(StringRef CPU, StringRef FS) {
HasStoreImm = !Disable_StoreImm;
HasLoadAcqStoreRel = !Disable_load_acq_store_rel;
HasGotox = !Disable_gotox;
+ HasHmul = !Disable_hmul;
return;
}
}
diff --git a/llvm/lib/Target/BPF/BPFSubtarget.h b/llvm/lib/Target/BPF/BPFSubtarget.h
index 24eff862224b04..4be57b424ece15 100644
--- a/llvm/lib/Target/BPF/BPFSubtarget.h
+++ b/llvm/lib/Target/BPF/BPFSubtarget.h
@@ -68,7 +68,7 @@ class BPFSubtarget : public BPFGenSubtargetInfo {
// whether cpu v4 insns are enabled.
bool HasLdsx, HasMovsx, HasBswap, HasSdivSmod, HasGotol, HasStoreImm,
- HasLoadAcqStoreRel, HasGotox;
+ HasLoadAcqStoreRel, HasGotox, HasHmul;
std::unique_ptr<CallLowering> CallLoweringInfo;
std::unique_ptr<InstructionSelector> InstSelector;
@@ -101,6 +101,7 @@ class BPFSubtarget : public BPFGenSubtargetInfo {
bool hasStoreImm() const { return HasStoreImm; }
bool hasLoadAcqStoreRel() const { return HasLoadAcqStoreRel; }
bool hasGotox() const { return HasGotox; }
+ bool hasHmul() const { return HasHmul; }
bool isLittleEndian() const { return IsLittleEndian; }
diff --git a/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s b/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s
index c4949d5d1df5e3..90b709dabe67f9 100644
--- a/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s
+++ b/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s
@@ -38,6 +38,15 @@ w2 = (s16)w4
r1 s/= r3
r2 s%= r4
+// CHECK: 2f 21 01 00 00 00 00 00 r1 uh*= r2
+// CHECK: 2f 43 02 00 00 00 00 00 r3 sh*= r4
+// CHECK: 27 05 01 00 fe ff ff ff r5 uh*= -0x2
+// CHECK: 27 06 02 00 fd ff ff ff r6 sh*= -0x3
+r1 uh*= r2
+r3 sh*= r4
+r5 uh*= -2
+r6 sh*= -3
+
// CHECK: 3c 31 01 00 00 00 00 00 w1 s/= w3
// CHECK: 9c 42 01 00 00 00 00 00 w2 s%= w4
w1 s/= w3
diff --git a/llvm/test/CodeGen/BPF/hmul.ll b/llvm/test/CodeGen/BPF/hmul.ll
new file mode 100644
index 00000000000000..43e5ad0e8b4e23
--- /dev/null
+++ b/llvm/test/CodeGen/BPF/hmul.ll
@@ -0,0 +1,83 @@
+; RUN: llc -mtriple=bpfel -mcpu=v4 -verify-machineinstrs -show-mc-encoding < %s | FileCheck %s --check-prefix=V4
+; RUN: llc -mtriple=bpfel -mcpu=v3 -verify-machineinstrs < %s | FileCheck %s --check-prefix=NO-HMUL
+; RUN: llc -mtriple=bpfel -mcpu=v4 -disable-hmul -verify-machineinstrs < %s | FileCheck %s --check-prefix=NO-HMUL
+
+define i64 @uhmul_rr(i64 %a, i64 %b) {
+; V4-LABEL: uhmul_rr:
+; V4: r0 = r1
+; V4-NEXT: r0 uh*= r2 # encoding: [0x2f,0x20,0x01,0x00,0x00,0x00,0x00,0x00]
+; V4-NEXT: exit
+; NO-HMUL-LABEL: uhmul_rr:
+; NO-HMUL-NOT: uh*=
+; NO-HMUL-NOT: sh*=
+; NO-HMUL: call __multi3
+; NO-HMUL: exit
+ %awide = zext i64 %a to i128
+ %bwide = zext i64 %b to i128
+ %product = mul i128 %awide, %bwide
+ %high = lshr i128 %product, 64
+ %result = trunc i128 %high to i64
+ ret i64 %result
+}
+
+define i64 @shmul_rr(i64 %a, i64 %b) {
+; V4-LABEL: shmul_rr:
+; V4: r0 = r1
+; V4-NEXT: r0 sh*= r2 # encoding: [0x2f,0x20,0x02,0x00,0x00,0x00,0x00,0x00]
+; V4-NEXT: exit
+; NO-HMUL-LABEL: shmul_rr:
+; NO-HMUL-NOT: uh*=
+; NO-HMUL-NOT: sh*=
+; NO-HMUL: call __multi3
+; NO-HMUL: exit
+ %awide = sext i64 %a to i128
+ %bwide = sext i64 %b to i128
+ %product = mul i128 %awide, %bwide
+ %high = ashr i128 %product, 64
+ %result = trunc i128 %high to i64
+ ret i64 %result
+}
+
+define i64 @uhmul_ri(i64 %a) {
+; V4-LABEL: uhmul_ri:
+; V4: r0 = r1
+; V4-NEXT: r0 uh*= -2 # encoding: [0x27,0x00,0x01,0x00,0xfe,0xff,0xff,0xff]
+; V4-NEXT: exit
+; NO-HMUL-LABEL: uhmul_ri:
+; NO-HMUL-NOT: uh*=
+; NO-HMUL-NOT: sh*=
+; NO-HMUL: call __multi3
+; NO-HMUL: exit
+ %awide = zext i64 %a to i128
+ %product = mul i128 %awide, 18446744073709551614
+ %high = lshr i128 %product, 64
+ %result = trunc i128 %high to i64
+ ret i64 %result
+}
+
+define i64 @shmul_ri(i64 %a) {
+; V4-LABEL: shmul_ri:
+; V4: r0 = r1
+; V4-NEXT: r0 sh*= -2 # encoding: [0x27,0x00,0x02,0x00,0xfe,0xff,0xff,0xff]
+; V4-NEXT: exit
+; NO-HMUL-LABEL: shmul_ri:
+; NO-HMUL-NOT: uh*=
+; NO-HMUL-NOT: sh*=
+; NO-HMUL: exit
+ %awide = sext i64 %a to i128
+ %product = mul i128 %awide, -2
+ %high = ashr i128 %product, 64
+ %result = trunc i128 %high to i64
+ ret i64 %result
+}
+
+define i64 @mul_low(i64 %a, i64 %b) {
+; V4-LABEL: mul_low:
+; V4: r0 = r1
+; V4-NEXT: r0 *= r2 # encoding: [0x2f,0x20,0x00,0x00,0x00,0x00,0x00,0x00]
+; V4-NEXT: exit
+; NO-HMUL-LABEL: mul_low:
+; NO-HMUL: r0 *= r2
+ %product = mul i64 %a, %b
+ ret i64 %product
+}
diff --git a/llvm/test/MC/BPF/bad-hmul.s b/llvm/test/MC/BPF/bad-hmul.s
new file mode 100644
index 00000000000000..e98e30f406c425
--- /dev/null
+++ b/llvm/test/MC/BPF/bad-hmul.s
@@ -0,0 +1,5 @@
+# RUN: not llvm-mc -mcpu=v4 -triple bpfel < %s 2>&1 \
+# RUN: | grep 'error: invalid operand for instruction' \
+# RUN: | count 2
+ w1 uh*= w2
+ w3 sh*= -2
>From bd113192ea18608fce7a1c549c0d3b0295702072 Mon Sep 17 00:00:00 2001
From: Yazhou Tang <tangyazhou518 at outlook.com>
Date: Tue, 20 Oct 2026 22:10:26 +0800
Subject: [PATCH 2/3] [BPF][test] Use implicit negative checks for high-half
multiply
Replace per-function NO-HMUL-NOT directives with FileCheck implicit negative checks for UHMUL and SHMUL. Check all intervals between positive matches in the v3 and v4 -disable-hmul runs.
---
llvm/test/CodeGen/BPF/hmul.ll | 14 ++++----------
1 file changed, 4 insertions(+), 10 deletions(-)
diff --git a/llvm/test/CodeGen/BPF/hmul.ll b/llvm/test/CodeGen/BPF/hmul.ll
index 43e5ad0e8b4e23..c44f29ced6aff7 100644
--- a/llvm/test/CodeGen/BPF/hmul.ll
+++ b/llvm/test/CodeGen/BPF/hmul.ll
@@ -1,6 +1,8 @@
; RUN: llc -mtriple=bpfel -mcpu=v4 -verify-machineinstrs -show-mc-encoding < %s | FileCheck %s --check-prefix=V4
-; RUN: llc -mtriple=bpfel -mcpu=v3 -verify-machineinstrs < %s | FileCheck %s --check-prefix=NO-HMUL
-; RUN: llc -mtriple=bpfel -mcpu=v4 -disable-hmul -verify-machineinstrs < %s | FileCheck %s --check-prefix=NO-HMUL
+; RUN: llc -mtriple=bpfel -mcpu=v3 -verify-machineinstrs < %s \
+; RUN: | FileCheck %s --check-prefix=NO-HMUL --implicit-check-not="uh*=" --implicit-check-not="sh*="
+; RUN: llc -mtriple=bpfel -mcpu=v4 -disable-hmul -verify-machineinstrs < %s \
+; RUN: | FileCheck %s --check-prefix=NO-HMUL --implicit-check-not="uh*=" --implicit-check-not="sh*="
define i64 @uhmul_rr(i64 %a, i64 %b) {
; V4-LABEL: uhmul_rr:
@@ -8,8 +10,6 @@ define i64 @uhmul_rr(i64 %a, i64 %b) {
; V4-NEXT: r0 uh*= r2 # encoding: [0x2f,0x20,0x01,0x00,0x00,0x00,0x00,0x00]
; V4-NEXT: exit
; NO-HMUL-LABEL: uhmul_rr:
-; NO-HMUL-NOT: uh*=
-; NO-HMUL-NOT: sh*=
; NO-HMUL: call __multi3
; NO-HMUL: exit
%awide = zext i64 %a to i128
@@ -26,8 +26,6 @@ define i64 @shmul_rr(i64 %a, i64 %b) {
; V4-NEXT: r0 sh*= r2 # encoding: [0x2f,0x20,0x02,0x00,0x00,0x00,0x00,0x00]
; V4-NEXT: exit
; NO-HMUL-LABEL: shmul_rr:
-; NO-HMUL-NOT: uh*=
-; NO-HMUL-NOT: sh*=
; NO-HMUL: call __multi3
; NO-HMUL: exit
%awide = sext i64 %a to i128
@@ -44,8 +42,6 @@ define i64 @uhmul_ri(i64 %a) {
; V4-NEXT: r0 uh*= -2 # encoding: [0x27,0x00,0x01,0x00,0xfe,0xff,0xff,0xff]
; V4-NEXT: exit
; NO-HMUL-LABEL: uhmul_ri:
-; NO-HMUL-NOT: uh*=
-; NO-HMUL-NOT: sh*=
; NO-HMUL: call __multi3
; NO-HMUL: exit
%awide = zext i64 %a to i128
@@ -61,8 +57,6 @@ define i64 @shmul_ri(i64 %a) {
; V4-NEXT: r0 sh*= -2 # encoding: [0x27,0x00,0x02,0x00,0xfe,0xff,0xff,0xff]
; V4-NEXT: exit
; NO-HMUL-LABEL: shmul_ri:
-; NO-HMUL-NOT: uh*=
-; NO-HMUL-NOT: sh*=
; NO-HMUL: exit
%awide = sext i64 %a to i128
%product = mul i128 %awide, -2
>From 550a84f6f59888e34d9b01167aa903a9acb96545 Mon Sep 17 00:00:00 2001
From: Yazhou Tang <tangyazhou518 at outlook.com>
Date: Tue, 20 Oct 2026 19:16:32 +0800
Subject: [PATCH 3/3] [BPF] Introduce cpu v5 for high-half multiply
Introduce -mcpu=v5 for UHMUL/SHMUL while preserving v4 code generation. Update Clang CPU support, feature macros, and tests, retaining -disable-hmul.
Suggested by Alexei Starovoitov: https://lore.kernel.org/bpf/DLHSD07GRG0W.16CFRV44ZE1DD@gmail.com/
---
clang/docs/ReleaseNotes.md | 5 ++
clang/lib/Basic/Targets/BPF.cpp | 7 ++-
clang/lib/Basic/Targets/BPF.h | 2 +-
clang/test/Misc/target-invalid-cpu-note/bpf.c | 1 +
.../test/Preprocessor/bpf-predefined-macros.c | 29 +++++++++++-
llvm/docs/ReleaseNotes.md | 6 +++
llvm/lib/Target/BPF/BPF.td | 1 +
llvm/lib/Target/BPF/BPFSubtarget.cpp | 4 +-
llvm/lib/Target/BPF/BPFSubtarget.h | 5 +-
.../CodeGen/BPF/assembler-disassembler-v4.s | 9 ----
.../CodeGen/BPF/assembler-disassembler-v5.s | 12 +++++
llvm/test/CodeGen/BPF/hmul.ll | 46 ++++++++++---------
llvm/test/CodeGen/BPF/sdiv_smod.ll | 1 +
llvm/test/MC/BPF/bad-hmul.s | 2 +-
14 files changed, 90 insertions(+), 40 deletions(-)
create mode 100644 llvm/test/CodeGen/BPF/assembler-disassembler-v5.s
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index 830109b8c4a399..f2105d09f73eef 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -1134,6 +1134,11 @@ features cannot lower the translation-unit ABI level;
- `__builtin_amdgcn_fcmp`
- `__builtin_amdgcn_fcmpf`
+#### BPF Support
+
+- Added `-mcpu=v5` and the `__BPF_FEATURE_HMUL` predefined macro for unsigned
+ and signed high-half multiply instructions.
+
#### DirectX Support
- `clang-dxc` and HLSL support are now enabled by default, following the
diff --git a/clang/lib/Basic/Targets/BPF.cpp b/clang/lib/Basic/Targets/BPF.cpp
index 6efdd023b32d14..2c37d56186ce52 100644
--- a/clang/lib/Basic/Targets/BPF.cpp
+++ b/clang/lib/Basic/Targets/BPF.cpp
@@ -78,12 +78,15 @@ void BPFTargetInfo::getTargetDefines(const LangOptions &Opts,
Builder.defineMacro("__BPF_FEATURE_ST");
Builder.defineMacro("__BPF_FEATURE_LOAD_ACQ_STORE_REL");
Builder.defineMacro("__BPF_FEATURE_GOTOX");
+ }
+
+ if (CpuVerNum >= 5) {
Builder.defineMacro("__BPF_FEATURE_HMUL");
}
}
-static constexpr llvm::StringLiteral ValidCPUNames[] = {"generic", "v1", "v2",
- "v3", "v4", "probe"};
+static constexpr llvm::StringLiteral ValidCPUNames[] = {
+ "generic", "v1", "v2", "v3", "v4", "v5", "probe"};
bool BPFTargetInfo::isValidCPUName(StringRef Name) const {
return llvm::is_contained(ValidCPUNames, Name);
diff --git a/clang/lib/Basic/Targets/BPF.h b/clang/lib/Basic/Targets/BPF.h
index e51f37b851d481..73ab60339a311a 100644
--- a/clang/lib/Basic/Targets/BPF.h
+++ b/clang/lib/Basic/Targets/BPF.h
@@ -100,7 +100,7 @@ class LLVM_LIBRARY_VISIBILITY BPFTargetInfo : public TargetInfo {
void fillValidCPUList(SmallVectorImpl<StringRef> &Values) const override;
bool setCPU(StringRef Name) override {
- if (Name == "v3" || Name == "v4") {
+ if (Name == "v3" || Name == "v4" || Name == "v5") {
HasAlu32 = true;
}
diff --git a/clang/test/Misc/target-invalid-cpu-note/bpf.c b/clang/test/Misc/target-invalid-cpu-note/bpf.c
index fe925f86cdd137..7f90e64ab16f16 100644
--- a/clang/test/Misc/target-invalid-cpu-note/bpf.c
+++ b/clang/test/Misc/target-invalid-cpu-note/bpf.c
@@ -10,6 +10,7 @@
// CHECK-SAME: {{^}}, v2
// CHECK-SAME: {{^}}, v3
// CHECK-SAME: {{^}}, v4
+// CHECK-SAME: {{^}}, v5
// CHECK-SAME: {{^}}, probe
// CHECK-SAME: {{$}}
diff --git a/clang/test/Preprocessor/bpf-predefined-macros.c b/clang/test/Preprocessor/bpf-predefined-macros.c
index c81bfa871a440d..323c566ffd8f19 100644
--- a/clang/test/Preprocessor/bpf-predefined-macros.c
+++ b/clang/test/Preprocessor/bpf-predefined-macros.c
@@ -4,6 +4,8 @@
// RUN: %clang -E -target bpfel -mcpu=v2 -x c -o - %s | FileCheck -check-prefix=CHECK -check-prefix=CPU_V2 %s
// RUN: %clang -E -target bpfel -mcpu=v3 -x c -o - %s | FileCheck -check-prefix=CHECK -check-prefix=CPU_V3 %s
// RUN: %clang -E -target bpfel -mcpu=v4 -x c -o - %s | FileCheck -check-prefix=CHECK -check-prefix=CPU_V4 %s
+// RUN: %clang -E -target bpfel -mcpu=v5 -x c -o - %s | FileCheck -check-prefix=CHECK -check-prefix=CPU_V5 %s
+// RUN: %clang -E -target bpfeb -mcpu=v5 -x c -o - %s | FileCheck -check-prefix=CHECK -check-prefix=CPU_V5 %s
// RUN: %clang -E -target bpfel -mcpu=generic -x c -o - %s | FileCheck -check-prefix=CHECK -check-prefix=CPU_GENERIC %s
// RUN: %clang -E -target bpfel -mcpu=probe -x c -o - %s | FileCheck -check-prefix=CHECK -check-prefix=CPU_PROBE %s
@@ -34,6 +36,9 @@ int i;
#if __BPF_CPU_VERSION__ == 4
int j;
#endif
+#if __BPF_CPU_VERSION__ == 5
+int aa;
+#endif
#ifdef __BPF_FEATURE_JMP_EXT
int k;
#endif
@@ -83,7 +88,10 @@ int y;
int z;
#endif
#ifdef __BPF_FEATURE_HMUL
-int aa;
+int ab;
+#endif
+#if __BPF_CPU_VERSION__ < 5 && defined(__BPF_FEATURE_HMUL)
+#error HMUL requires CPU v5 or later
#endif
// CHECK: int b;
@@ -114,35 +122,52 @@ int aa;
// CPU_V4: int r;
// CPU_V4: int s;
+// CPU_V5: int aa;
+// CPU_V5: int k;
+// CPU_V5: int l;
+// CPU_V5: int m;
+// CPU_V5: int n;
+// CPU_V5: int o;
+// CPU_V5: int p;
+// CPU_V5: int q;
+// CPU_V5: int r;
+// CPU_V5: int s;
+
// CPU_V1: int t;
// CPU_V2: int t;
// CPU_V3: int t;
// CPU_V4: int t;
+// CPU_V5: int t;
// CPU_V1: int u;
// CPU_V2: int u;
// CPU_V3: int u;
// CPU_V4: int u;
+// CPU_V5: int u;
// CPU_V4: int v;
// CPU_V4: int w;
+// CPU_V5: int v;
+// CPU_V5: int w;
// CPU_V1: int x;
// CPU_V2: int x;
// CPU_V3: int x;
// CPU_V4: int x;
+// CPU_V5: int x;
// CPU_V1: int y;
// CPU_V2: int y;
// CPU_V3: int y;
// CPU_V4: int y;
+// CPU_V5: int y;
// CPU_V1: int z;
// CPU_V2: int z;
// CPU_V3: int z;
// CPU_V4: int z;
-// CPU_V4: int aa;
+// CPU_V5: int ab;
// CPU_GENERIC: int g;
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 77d639d218ef20..9868f7fda4d44e 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -296,6 +296,12 @@ Makes programs 10x faster by doing Special New Thing.
### Changes to the AVR Backend
+### Changes to the BPF Backend
+
+* Added `-mcpu=v5`, which includes the v4 instruction set and enables unsigned
+ and signed high-half multiply instructions (`UHMUL` and `SHMUL`). Use
+ `-disable-hmul` to disable their generation.
+
### Changes to the DirectX Backend
* The DirectX backend has been promoted from experimental to an official,
diff --git a/llvm/lib/Target/BPF/BPF.td b/llvm/lib/Target/BPF/BPF.td
index 1fc364dad9988f..f52c8eadf60fd2 100644
--- a/llvm/lib/Target/BPF/BPF.td
+++ b/llvm/lib/Target/BPF/BPF.td
@@ -39,6 +39,7 @@ def : Proc<"v1", []>;
def : Proc<"v2", []>;
def : Proc<"v3", [ALU32]>;
def : Proc<"v4", [ALU32]>;
+def : Proc<"v5", [ALU32]>;
def : Proc<"probe", []>;
def BPFInstPrinter : AsmWriter {
diff --git a/llvm/lib/Target/BPF/BPFSubtarget.cpp b/llvm/lib/Target/BPF/BPFSubtarget.cpp
index 4e9a19a235300a..dd8201e3b122b7 100644
--- a/llvm/lib/Target/BPF/BPFSubtarget.cpp
+++ b/llvm/lib/Target/BPF/BPFSubtarget.cpp
@@ -93,7 +93,7 @@ void BPFSubtarget::initSubtargetFeatures(StringRef CPU, StringRef FS) {
HasAlu32 = true;
return;
}
- if (CPU == "v4") {
+ if (CPU == "v4" || CPU == "v5") {
HasJmpExt = true;
HasJmp32 = true;
HasAlu32 = true;
@@ -105,7 +105,7 @@ void BPFSubtarget::initSubtargetFeatures(StringRef CPU, StringRef FS) {
HasStoreImm = !Disable_StoreImm;
HasLoadAcqStoreRel = !Disable_load_acq_store_rel;
HasGotox = !Disable_gotox;
- HasHmul = !Disable_hmul;
+ HasHmul = CPU == "v5" && !Disable_hmul;
return;
}
}
diff --git a/llvm/lib/Target/BPF/BPFSubtarget.h b/llvm/lib/Target/BPF/BPFSubtarget.h
index 4be57b424ece15..f7808facf0822e 100644
--- a/llvm/lib/Target/BPF/BPFSubtarget.h
+++ b/llvm/lib/Target/BPF/BPFSubtarget.h
@@ -68,7 +68,10 @@ class BPFSubtarget : public BPFGenSubtargetInfo {
// whether cpu v4 insns are enabled.
bool HasLdsx, HasMovsx, HasBswap, HasSdivSmod, HasGotol, HasStoreImm,
- HasLoadAcqStoreRel, HasGotox, HasHmul;
+ HasLoadAcqStoreRel, HasGotox;
+
+ // whether cpu v5 insns are enabled.
+ bool HasHmul;
std::unique_ptr<CallLowering> CallLoweringInfo;
std::unique_ptr<InstructionSelector> InstSelector;
diff --git a/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s b/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s
index 90b709dabe67f9..c4949d5d1df5e3 100644
--- a/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s
+++ b/llvm/test/CodeGen/BPF/assembler-disassembler-v4.s
@@ -38,15 +38,6 @@ w2 = (s16)w4
r1 s/= r3
r2 s%= r4
-// CHECK: 2f 21 01 00 00 00 00 00 r1 uh*= r2
-// CHECK: 2f 43 02 00 00 00 00 00 r3 sh*= r4
-// CHECK: 27 05 01 00 fe ff ff ff r5 uh*= -0x2
-// CHECK: 27 06 02 00 fd ff ff ff r6 sh*= -0x3
-r1 uh*= r2
-r3 sh*= r4
-r5 uh*= -2
-r6 sh*= -3
-
// CHECK: 3c 31 01 00 00 00 00 00 w1 s/= w3
// CHECK: 9c 42 01 00 00 00 00 00 w2 s%= w4
w1 s/= w3
diff --git a/llvm/test/CodeGen/BPF/assembler-disassembler-v5.s b/llvm/test/CodeGen/BPF/assembler-disassembler-v5.s
new file mode 100644
index 00000000000000..e51e0c7f3f6964
--- /dev/null
+++ b/llvm/test/CodeGen/BPF/assembler-disassembler-v5.s
@@ -0,0 +1,12 @@
+// RUN: llvm-mc -triple bpfel --mcpu=v5 --assemble --filetype=obj %s \
+// RUN: | llvm-objdump -d --mattr=+alu32 - \
+// RUN: | FileCheck %s
+
+// CHECK: 2f 21 01 00 00 00 00 00 r1 uh*= r2
+// CHECK: 2f 43 02 00 00 00 00 00 r3 sh*= r4
+// CHECK: 27 05 01 00 fe ff ff ff r5 uh*= -0x2
+// CHECK: 27 06 02 00 fd ff ff ff r6 sh*= -0x3
+r1 uh*= r2
+r3 sh*= r4
+r5 uh*= -2
+r6 sh*= -3
diff --git a/llvm/test/CodeGen/BPF/hmul.ll b/llvm/test/CodeGen/BPF/hmul.ll
index c44f29ced6aff7..42d358f9f4df37 100644
--- a/llvm/test/CodeGen/BPF/hmul.ll
+++ b/llvm/test/CodeGen/BPF/hmul.ll
@@ -1,14 +1,16 @@
-; RUN: llc -mtriple=bpfel -mcpu=v4 -verify-machineinstrs -show-mc-encoding < %s | FileCheck %s --check-prefix=V4
+; RUN: llc -mtriple=bpfel -mcpu=v5 -verify-machineinstrs -show-mc-encoding < %s | FileCheck %s --check-prefix=V5
; RUN: llc -mtriple=bpfel -mcpu=v3 -verify-machineinstrs < %s \
; RUN: | FileCheck %s --check-prefix=NO-HMUL --implicit-check-not="uh*=" --implicit-check-not="sh*="
-; RUN: llc -mtriple=bpfel -mcpu=v4 -disable-hmul -verify-machineinstrs < %s \
+; RUN: llc -mtriple=bpfel -mcpu=v4 -verify-machineinstrs < %s \
+; RUN: | FileCheck %s --check-prefix=NO-HMUL --implicit-check-not="uh*=" --implicit-check-not="sh*="
+; RUN: llc -mtriple=bpfel -mcpu=v5 -disable-hmul -verify-machineinstrs < %s \
; RUN: | FileCheck %s --check-prefix=NO-HMUL --implicit-check-not="uh*=" --implicit-check-not="sh*="
define i64 @uhmul_rr(i64 %a, i64 %b) {
-; V4-LABEL: uhmul_rr:
-; V4: r0 = r1
-; V4-NEXT: r0 uh*= r2 # encoding: [0x2f,0x20,0x01,0x00,0x00,0x00,0x00,0x00]
-; V4-NEXT: exit
+; V5-LABEL: uhmul_rr:
+; V5: r0 = r1
+; V5-NEXT: r0 uh*= r2 # encoding: [0x2f,0x20,0x01,0x00,0x00,0x00,0x00,0x00]
+; V5-NEXT: exit
; NO-HMUL-LABEL: uhmul_rr:
; NO-HMUL: call __multi3
; NO-HMUL: exit
@@ -21,10 +23,10 @@ define i64 @uhmul_rr(i64 %a, i64 %b) {
}
define i64 @shmul_rr(i64 %a, i64 %b) {
-; V4-LABEL: shmul_rr:
-; V4: r0 = r1
-; V4-NEXT: r0 sh*= r2 # encoding: [0x2f,0x20,0x02,0x00,0x00,0x00,0x00,0x00]
-; V4-NEXT: exit
+; V5-LABEL: shmul_rr:
+; V5: r0 = r1
+; V5-NEXT: r0 sh*= r2 # encoding: [0x2f,0x20,0x02,0x00,0x00,0x00,0x00,0x00]
+; V5-NEXT: exit
; NO-HMUL-LABEL: shmul_rr:
; NO-HMUL: call __multi3
; NO-HMUL: exit
@@ -37,10 +39,10 @@ define i64 @shmul_rr(i64 %a, i64 %b) {
}
define i64 @uhmul_ri(i64 %a) {
-; V4-LABEL: uhmul_ri:
-; V4: r0 = r1
-; V4-NEXT: r0 uh*= -2 # encoding: [0x27,0x00,0x01,0x00,0xfe,0xff,0xff,0xff]
-; V4-NEXT: exit
+; V5-LABEL: uhmul_ri:
+; V5: r0 = r1
+; V5-NEXT: r0 uh*= -2 # encoding: [0x27,0x00,0x01,0x00,0xfe,0xff,0xff,0xff]
+; V5-NEXT: exit
; NO-HMUL-LABEL: uhmul_ri:
; NO-HMUL: call __multi3
; NO-HMUL: exit
@@ -52,10 +54,10 @@ define i64 @uhmul_ri(i64 %a) {
}
define i64 @shmul_ri(i64 %a) {
-; V4-LABEL: shmul_ri:
-; V4: r0 = r1
-; V4-NEXT: r0 sh*= -2 # encoding: [0x27,0x00,0x02,0x00,0xfe,0xff,0xff,0xff]
-; V4-NEXT: exit
+; V5-LABEL: shmul_ri:
+; V5: r0 = r1
+; V5-NEXT: r0 sh*= -2 # encoding: [0x27,0x00,0x02,0x00,0xfe,0xff,0xff,0xff]
+; V5-NEXT: exit
; NO-HMUL-LABEL: shmul_ri:
; NO-HMUL: exit
%awide = sext i64 %a to i128
@@ -66,10 +68,10 @@ define i64 @shmul_ri(i64 %a) {
}
define i64 @mul_low(i64 %a, i64 %b) {
-; V4-LABEL: mul_low:
-; V4: r0 = r1
-; V4-NEXT: r0 *= r2 # encoding: [0x2f,0x20,0x00,0x00,0x00,0x00,0x00,0x00]
-; V4-NEXT: exit
+; V5-LABEL: mul_low:
+; V5: r0 = r1
+; V5-NEXT: r0 *= r2 # encoding: [0x2f,0x20,0x00,0x00,0x00,0x00,0x00,0x00]
+; V5-NEXT: exit
; NO-HMUL-LABEL: mul_low:
; NO-HMUL: r0 *= r2
%product = mul i64 %a, %b
diff --git a/llvm/test/CodeGen/BPF/sdiv_smod.ll b/llvm/test/CodeGen/BPF/sdiv_smod.ll
index 5fb23d03f172db..6d921d683c66c3 100644
--- a/llvm/test/CodeGen/BPF/sdiv_smod.ll
+++ b/llvm/test/CodeGen/BPF/sdiv_smod.ll
@@ -1,4 +1,5 @@
; RUN: llc -mtriple=bpfel -mcpu=v4 -verify-machineinstrs -show-mc-encoding < %s | FileCheck %s
+; RUN: llc -mtriple=bpfel -mcpu=v5 -verify-machineinstrs -show-mc-encoding < %s | FileCheck %s
; Source:
; int foo(int a, int b, int c) {
; return a/b + a%c;
diff --git a/llvm/test/MC/BPF/bad-hmul.s b/llvm/test/MC/BPF/bad-hmul.s
index e98e30f406c425..ae163662eefea7 100644
--- a/llvm/test/MC/BPF/bad-hmul.s
+++ b/llvm/test/MC/BPF/bad-hmul.s
@@ -1,4 +1,4 @@
-# RUN: not llvm-mc -mcpu=v4 -triple bpfel < %s 2>&1 \
+# RUN: not llvm-mc -mcpu=v5 -triple bpfel < %s 2>&1 \
# RUN: | grep 'error: invalid operand for instruction' \
# RUN: | count 2
w1 uh*= w2
More information about the cfe-commits
mailing list