[llvm] be6fe95 - [RISCV] Implement a proposed mapping for Ztso
Philip Reames via llvm-commits
llvm-commits at lists.llvm.org
Thu Feb 9 12:00:43 PST 2023
Author: Philip Reames
Date: 2023-02-09T12:00:05-08:00
New Revision: be6fe95673624f4faec4f7ed437842041b95b1e9
URL: https://github.com/llvm/llvm-project/commit/be6fe95673624f4faec4f7ed437842041b95b1e9
DIFF: https://github.com/llvm/llvm-project/commit/be6fe95673624f4faec4f7ed437842041b95b1e9.diff
LOG: [RISCV] Implement a proposed mapping for Ztso
This change implements a proposed lowering from LLVM's memory model to the TSO memory model defined by the Ztso extension. Selecting the proposed mapping turns out to be an involved conversation that really didn't fit within a review description, so let me refer you to https://github.com/preames/public-notes/blob/master/riscv-tso-mappings.rst. This review implements the WMO compatible variant (the proposed one in that document).
Ztso is currently accepted as an experimental extension in LLVM. Despite the fact the extension was recently ratified, I think we need to leave it as experimental until we have wide agreement on the chosen mapping for ABI purposes.
I need to note that the current in-tree implementation defaults to generating WMO compatible fences. This is entirely compatible with the proposed mapping in this patch, but is unfortunately not compatible with the major alternative. The in tree implementation is explicitly experimental so the impact of this is limited, but it is worth calling out that if settle on the alternative we will have a minor ABI break. My apologies for not calling this out in the original patch; I had not realized at the time that one of our realistic choices for mappings wouldn't be WMO compatible.
This patch only contains the changes for load/store and fence. That is, it does not change the lowering for atomicrmw operations. This is a sound thing to do under the proposed mapping since the existing WMO mappings remain compatible. I do plan to change these; I'm just working incrementally.
Differential Revision: https://reviews.llvm.org/D143076
Added:
Modified:
llvm/docs/RISCVUsage.rst
llvm/lib/Target/RISCV/RISCVISelLowering.cpp
llvm/test/CodeGen/RISCV/atomic-fence.ll
llvm/test/CodeGen/RISCV/atomic-load-store.ll
Removed:
################################################################################
diff --git a/llvm/docs/RISCVUsage.rst b/llvm/docs/RISCVUsage.rst
index 251091788846d..2a79792a94d15 100644
--- a/llvm/docs/RISCVUsage.rst
+++ b/llvm/docs/RISCVUsage.rst
@@ -151,7 +151,7 @@ The primary goal of experimental support is to assist in the process of ratifica
LLVM implements the `0.2 draft specification <https://github.com/riscv/riscv-isa-manual/releases/tag/draft-20220831-bf5a151>`_.
``experimental-ztso``
- LLVM implements the `v0.1 proposed specification <https://github.com/riscv/riscv-isa-manual/releases/download/draft-20220723-10eea63/riscv-spec.pdf>`_ (see Chapter 25). Using will set appropriate ELF flags and attributes, but does not yet change code generation.
+ LLVM implements the `v0.1 proposed specification <https://github.com/riscv/riscv-isa-manual/releases/download/draft-20220723-10eea63/riscv-spec.pdf>`_ (see Chapter 25). The mapping from the C/C++ memory model to Ztso has not yet been ratified in any standards document. There are multiple possible mappings, and they are *not* mutually ABI compatible. The mapping LLVM implements is ABI compatible with the default WMO mapping. This mapping may change and there is *explicitly* no ABI stability offered while the extension remains in experimental status. User beware.
``experimental-zvfh``
LLVM implements `this draft text <https://github.com/riscv/riscv-v-spec/pull/780>`_.
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index c2eb508006697..5ac8884aab229 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -3721,11 +3721,25 @@ static SDValue lowerConstant(SDValue Op, SelectionDAG &DAG,
return SDValue();
}
-static SDValue LowerATOMIC_FENCE(SDValue Op, SelectionDAG &DAG) {
+static SDValue LowerATOMIC_FENCE(SDValue Op, SelectionDAG &DAG,
+ const RISCVSubtarget &Subtarget) {
SDLoc dl(Op);
+ AtomicOrdering FenceOrdering =
+ static_cast<AtomicOrdering>(Op.getConstantOperandVal(1));
SyncScope::ID FenceSSID =
static_cast<SyncScope::ID>(Op.getConstantOperandVal(2));
+ if (Subtarget.hasStdExtZtso()) {
+ // The only fence that needs an instruction is a sequentially-consistent
+ // cross-thread fence.
+ if (FenceOrdering == AtomicOrdering::SequentiallyConsistent &&
+ FenceSSID == SyncScope::System)
+ return Op;
+
+ // MEMBARRIER is a compiler barrier; it codegens to a no-op.
+ return DAG.getNode(ISD::MEMBARRIER, dl, MVT::Other, Op.getOperand(0));
+ }
+
// singlethread fences only synchronize with signal handlers on the same
// thread and thus only need to preserve instruction order, not actually
// enforce memory ordering.
@@ -3742,7 +3756,7 @@ SDValue RISCVTargetLowering::LowerOperation(SDValue Op,
default:
report_fatal_error("unimplemented operand");
case ISD::ATOMIC_FENCE:
- return LowerATOMIC_FENCE(Op, DAG);
+ return LowerATOMIC_FENCE(Op, DAG, Subtarget);
case ISD::GlobalAddress:
return lowerGlobalAddress(Op, DAG);
case ISD::BlockAddress:
@@ -13759,6 +13773,12 @@ void RISCVTargetLowering::LowerAsmOperandForConstraint(
Instruction *RISCVTargetLowering::emitLeadingFence(IRBuilderBase &Builder,
Instruction *Inst,
AtomicOrdering Ord) const {
+ if (Subtarget.hasStdExtZtso()) {
+ if (isa<LoadInst>(Inst) && Ord == AtomicOrdering::SequentiallyConsistent)
+ return Builder.CreateFence(Ord);
+ return nullptr;
+ }
+
if (isa<LoadInst>(Inst) && Ord == AtomicOrdering::SequentiallyConsistent)
return Builder.CreateFence(Ord);
if (isa<StoreInst>(Inst) && isReleaseOrStronger(Ord))
@@ -13769,6 +13789,9 @@ Instruction *RISCVTargetLowering::emitLeadingFence(IRBuilderBase &Builder,
Instruction *RISCVTargetLowering::emitTrailingFence(IRBuilderBase &Builder,
Instruction *Inst,
AtomicOrdering Ord) const {
+ if (Subtarget.hasStdExtZtso())
+ return nullptr;
+
if (isa<LoadInst>(Inst) && isAcquireOrStronger(Ord))
return Builder.CreateFence(AtomicOrdering::Acquire);
return nullptr;
diff --git a/llvm/test/CodeGen/RISCV/atomic-fence.ll b/llvm/test/CodeGen/RISCV/atomic-fence.ll
index a8d49ce7fa24b..07e0cd066331a 100644
--- a/llvm/test/CodeGen/RISCV/atomic-fence.ll
+++ b/llvm/test/CodeGen/RISCV/atomic-fence.ll
@@ -1,40 +1,55 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
-; RUN: | FileCheck %s
+; RUN: | FileCheck --check-prefixes=CHECK,WMO %s
; RUN: llc -mtriple=riscv32 -mattr=+a -verify-machineinstrs < %s \
-; RUN: | FileCheck %s
+; RUN: | FileCheck --check-prefixes=CHECK,WMO %s
; RUN: llc -mtriple=riscv32 -mattr=+a,+experimental-ztso -verify-machineinstrs < %s \
-; RUN: | FileCheck %s
+; RUN: | FileCheck --check-prefixes=CHECK,TSO %s
; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
-; RUN: | FileCheck %s
+; RUN: | FileCheck --check-prefixes=CHECK,WMO %s
; RUN: llc -mtriple=riscv64 -mattr=+a -verify-machineinstrs < %s \
-; RUN: | FileCheck %s
+; RUN: | FileCheck --check-prefixes=CHECK,WMO %s
; RUN: llc -mtriple=riscv64 -mattr=+a,+experimental-ztso -verify-machineinstrs < %s \
-; RUN: | FileCheck %s
+; RUN: | FileCheck --check-prefixes=CHECK,TSO %s
define void @fence_acquire() nounwind {
-; CHECK-LABEL: fence_acquire:
-; CHECK: # %bb.0:
-; CHECK-NEXT: fence r, rw
-; CHECK-NEXT: ret
+; WMO-LABEL: fence_acquire:
+; WMO: # %bb.0:
+; WMO-NEXT: fence r, rw
+; WMO-NEXT: ret
+;
+; TSO-LABEL: fence_acquire:
+; TSO: # %bb.0:
+; TSO-NEXT: #MEMBARRIER
+; TSO-NEXT: ret
fence acquire
ret void
}
define void @fence_release() nounwind {
-; CHECK-LABEL: fence_release:
-; CHECK: # %bb.0:
-; CHECK-NEXT: fence rw, w
-; CHECK-NEXT: ret
+; WMO-LABEL: fence_release:
+; WMO: # %bb.0:
+; WMO-NEXT: fence rw, w
+; WMO-NEXT: ret
+;
+; TSO-LABEL: fence_release:
+; TSO: # %bb.0:
+; TSO-NEXT: #MEMBARRIER
+; TSO-NEXT: ret
fence release
ret void
}
define void @fence_acq_rel() nounwind {
-; CHECK-LABEL: fence_acq_rel:
-; CHECK: # %bb.0:
-; CHECK-NEXT: fence.tso
-; CHECK-NEXT: ret
+; WMO-LABEL: fence_acq_rel:
+; WMO: # %bb.0:
+; WMO-NEXT: fence.tso
+; WMO-NEXT: ret
+;
+; TSO-LABEL: fence_acq_rel:
+; TSO: # %bb.0:
+; TSO-NEXT: #MEMBARRIER
+; TSO-NEXT: ret
fence acq_rel
ret void
}
diff --git a/llvm/test/CodeGen/RISCV/atomic-load-store.ll b/llvm/test/CodeGen/RISCV/atomic-load-store.ll
index ff522689c3b9f..8eaa98bf33ccf 100644
--- a/llvm/test/CodeGen/RISCV/atomic-load-store.ll
+++ b/llvm/test/CodeGen/RISCV/atomic-load-store.ll
@@ -2,15 +2,15 @@
; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
; RUN: | FileCheck -check-prefix=RV32I %s
; RUN: llc -mtriple=riscv32 -mattr=+a -verify-machineinstrs < %s \
-; RUN: | FileCheck -check-prefix=RV32IA %s
+; RUN: | FileCheck -check-prefixes=RV32IA,RV32IA-WMO %s
; RUN: llc -mtriple=riscv32 -mattr=+a,+experimental-ztso -verify-machineinstrs < %s \
-; RUN: | FileCheck -check-prefix=RV32IA %s
+; RUN: | FileCheck -check-prefixes=RV32IA,RV32IA-TSO %s
; RUN: llc -mtriple=riscv64 -verify-machineinstrs < %s \
; RUN: | FileCheck -check-prefix=RV64I %s
; RUN: llc -mtriple=riscv64 -mattr=+a -verify-machineinstrs < %s \
-; RUN: | FileCheck -check-prefix=RV64IA %s
+; RUN: | FileCheck -check-prefixes=RV64IA,RV64IA-WMO %s
; RUN: llc -mtriple=riscv64 -mattr=+a,+experimental-ztso -verify-machineinstrs < %s \
-; RUN: | FileCheck -check-prefix=RV64IA %s
+; RUN: | FileCheck -check-prefixes=RV64IA,RV64IA-TSO %s
define i8 @atomic_load_i8_unordered(ptr %a) nounwind {
; RV32I-LABEL: atomic_load_i8_unordered:
@@ -91,11 +91,16 @@ define i8 @atomic_load_i8_acquire(ptr %a) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_load_i8_acquire:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: lb a0, 0(a0)
-; RV32IA-NEXT: fence r, rw
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_load_i8_acquire:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: lb a0, 0(a0)
+; RV32IA-WMO-NEXT: fence r, rw
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_load_i8_acquire:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: lb a0, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_load_i8_acquire:
; RV64I: # %bb.0:
@@ -107,11 +112,16 @@ define i8 @atomic_load_i8_acquire(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i8_acquire:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: lb a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i8_acquire:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: lb a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i8_acquire:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: lb a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i8, ptr %a acquire, align 1
ret i8 %1
}
@@ -127,12 +137,18 @@ define i8 @atomic_load_i8_seq_cst(ptr %a) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_load_i8_seq_cst:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, rw
-; RV32IA-NEXT: lb a0, 0(a0)
-; RV32IA-NEXT: fence r, rw
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_load_i8_seq_cst:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, rw
+; RV32IA-WMO-NEXT: lb a0, 0(a0)
+; RV32IA-WMO-NEXT: fence r, rw
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_load_i8_seq_cst:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: fence rw, rw
+; RV32IA-TSO-NEXT: lb a0, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_load_i8_seq_cst:
; RV64I: # %bb.0:
@@ -144,12 +160,18 @@ define i8 @atomic_load_i8_seq_cst(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i8_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, rw
-; RV64IA-NEXT: lb a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i8_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, rw
+; RV64IA-WMO-NEXT: lb a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i8_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: fence rw, rw
+; RV64IA-TSO-NEXT: lb a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i8, ptr %a seq_cst, align 1
ret i8 %1
}
@@ -233,11 +255,16 @@ define i16 @atomic_load_i16_acquire(ptr %a) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_load_i16_acquire:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: lh a0, 0(a0)
-; RV32IA-NEXT: fence r, rw
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_load_i16_acquire:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: lh a0, 0(a0)
+; RV32IA-WMO-NEXT: fence r, rw
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_load_i16_acquire:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: lh a0, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_load_i16_acquire:
; RV64I: # %bb.0:
@@ -249,11 +276,16 @@ define i16 @atomic_load_i16_acquire(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i16_acquire:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: lh a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i16_acquire:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: lh a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i16_acquire:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: lh a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i16, ptr %a acquire, align 2
ret i16 %1
}
@@ -269,12 +301,18 @@ define i16 @atomic_load_i16_seq_cst(ptr %a) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_load_i16_seq_cst:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, rw
-; RV32IA-NEXT: lh a0, 0(a0)
-; RV32IA-NEXT: fence r, rw
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_load_i16_seq_cst:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, rw
+; RV32IA-WMO-NEXT: lh a0, 0(a0)
+; RV32IA-WMO-NEXT: fence r, rw
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_load_i16_seq_cst:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: fence rw, rw
+; RV32IA-TSO-NEXT: lh a0, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_load_i16_seq_cst:
; RV64I: # %bb.0:
@@ -286,12 +324,18 @@ define i16 @atomic_load_i16_seq_cst(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i16_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, rw
-; RV64IA-NEXT: lh a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i16_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, rw
+; RV64IA-WMO-NEXT: lh a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i16_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: fence rw, rw
+; RV64IA-TSO-NEXT: lh a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i16, ptr %a seq_cst, align 2
ret i16 %1
}
@@ -375,11 +419,16 @@ define i32 @atomic_load_i32_acquire(ptr %a) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_load_i32_acquire:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: lw a0, 0(a0)
-; RV32IA-NEXT: fence r, rw
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_load_i32_acquire:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: lw a0, 0(a0)
+; RV32IA-WMO-NEXT: fence r, rw
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_load_i32_acquire:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: lw a0, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_load_i32_acquire:
; RV64I: # %bb.0:
@@ -391,11 +440,16 @@ define i32 @atomic_load_i32_acquire(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i32_acquire:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: lw a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i32_acquire:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: lw a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i32_acquire:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: lw a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i32, ptr %a acquire, align 4
ret i32 %1
}
@@ -411,12 +465,18 @@ define i32 @atomic_load_i32_seq_cst(ptr %a) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_load_i32_seq_cst:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, rw
-; RV32IA-NEXT: lw a0, 0(a0)
-; RV32IA-NEXT: fence r, rw
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_load_i32_seq_cst:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, rw
+; RV32IA-WMO-NEXT: lw a0, 0(a0)
+; RV32IA-WMO-NEXT: fence r, rw
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_load_i32_seq_cst:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: fence rw, rw
+; RV32IA-TSO-NEXT: lw a0, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_load_i32_seq_cst:
; RV64I: # %bb.0:
@@ -428,12 +488,18 @@ define i32 @atomic_load_i32_seq_cst(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i32_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, rw
-; RV64IA-NEXT: lw a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i32_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, rw
+; RV64IA-WMO-NEXT: lw a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i32_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: fence rw, rw
+; RV64IA-TSO-NEXT: lw a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i32, ptr %a seq_cst, align 4
ret i32 %1
}
@@ -547,11 +613,16 @@ define i64 @atomic_load_i64_acquire(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i64_acquire:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: ld a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i64_acquire:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: ld a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i64_acquire:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: ld a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i64, ptr %a acquire, align 8
ret i64 %1
}
@@ -587,12 +658,18 @@ define i64 @atomic_load_i64_seq_cst(ptr %a) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_load_i64_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, rw
-; RV64IA-NEXT: ld a0, 0(a0)
-; RV64IA-NEXT: fence r, rw
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_load_i64_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, rw
+; RV64IA-WMO-NEXT: ld a0, 0(a0)
+; RV64IA-WMO-NEXT: fence r, rw
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_load_i64_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: fence rw, rw
+; RV64IA-TSO-NEXT: ld a0, 0(a0)
+; RV64IA-TSO-NEXT: ret
%1 = load atomic i64, ptr %a seq_cst, align 8
ret i64 %1
}
@@ -676,11 +753,16 @@ define void @atomic_store_i8_release(ptr %a, i8 %b) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_store_i8_release:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, w
-; RV32IA-NEXT: sb a1, 0(a0)
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_store_i8_release:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, w
+; RV32IA-WMO-NEXT: sb a1, 0(a0)
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_store_i8_release:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: sb a1, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_store_i8_release:
; RV64I: # %bb.0:
@@ -692,11 +774,16 @@ define void @atomic_store_i8_release(ptr %a, i8 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i8_release:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sb a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i8_release:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sb a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i8_release:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sb a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i8 %b, ptr %a release, align 1
ret void
}
@@ -712,11 +799,16 @@ define void @atomic_store_i8_seq_cst(ptr %a, i8 %b) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_store_i8_seq_cst:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, w
-; RV32IA-NEXT: sb a1, 0(a0)
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_store_i8_seq_cst:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, w
+; RV32IA-WMO-NEXT: sb a1, 0(a0)
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_store_i8_seq_cst:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: sb a1, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_store_i8_seq_cst:
; RV64I: # %bb.0:
@@ -728,11 +820,16 @@ define void @atomic_store_i8_seq_cst(ptr %a, i8 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i8_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sb a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i8_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sb a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i8_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sb a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i8 %b, ptr %a seq_cst, align 1
ret void
}
@@ -816,11 +913,16 @@ define void @atomic_store_i16_release(ptr %a, i16 %b) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_store_i16_release:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, w
-; RV32IA-NEXT: sh a1, 0(a0)
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_store_i16_release:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, w
+; RV32IA-WMO-NEXT: sh a1, 0(a0)
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_store_i16_release:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: sh a1, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_store_i16_release:
; RV64I: # %bb.0:
@@ -832,11 +934,16 @@ define void @atomic_store_i16_release(ptr %a, i16 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i16_release:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sh a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i16_release:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sh a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i16_release:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sh a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i16 %b, ptr %a release, align 2
ret void
}
@@ -852,11 +959,16 @@ define void @atomic_store_i16_seq_cst(ptr %a, i16 %b) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_store_i16_seq_cst:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, w
-; RV32IA-NEXT: sh a1, 0(a0)
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_store_i16_seq_cst:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, w
+; RV32IA-WMO-NEXT: sh a1, 0(a0)
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_store_i16_seq_cst:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: sh a1, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_store_i16_seq_cst:
; RV64I: # %bb.0:
@@ -868,11 +980,16 @@ define void @atomic_store_i16_seq_cst(ptr %a, i16 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i16_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sh a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i16_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sh a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i16_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sh a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i16 %b, ptr %a seq_cst, align 2
ret void
}
@@ -956,11 +1073,16 @@ define void @atomic_store_i32_release(ptr %a, i32 %b) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_store_i32_release:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, w
-; RV32IA-NEXT: sw a1, 0(a0)
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_store_i32_release:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, w
+; RV32IA-WMO-NEXT: sw a1, 0(a0)
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_store_i32_release:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: sw a1, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_store_i32_release:
; RV64I: # %bb.0:
@@ -972,11 +1094,16 @@ define void @atomic_store_i32_release(ptr %a, i32 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i32_release:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sw a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i32_release:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sw a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i32_release:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sw a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i32 %b, ptr %a release, align 4
ret void
}
@@ -992,11 +1119,16 @@ define void @atomic_store_i32_seq_cst(ptr %a, i32 %b) nounwind {
; RV32I-NEXT: addi sp, sp, 16
; RV32I-NEXT: ret
;
-; RV32IA-LABEL: atomic_store_i32_seq_cst:
-; RV32IA: # %bb.0:
-; RV32IA-NEXT: fence rw, w
-; RV32IA-NEXT: sw a1, 0(a0)
-; RV32IA-NEXT: ret
+; RV32IA-WMO-LABEL: atomic_store_i32_seq_cst:
+; RV32IA-WMO: # %bb.0:
+; RV32IA-WMO-NEXT: fence rw, w
+; RV32IA-WMO-NEXT: sw a1, 0(a0)
+; RV32IA-WMO-NEXT: ret
+;
+; RV32IA-TSO-LABEL: atomic_store_i32_seq_cst:
+; RV32IA-TSO: # %bb.0:
+; RV32IA-TSO-NEXT: sw a1, 0(a0)
+; RV32IA-TSO-NEXT: ret
;
; RV64I-LABEL: atomic_store_i32_seq_cst:
; RV64I: # %bb.0:
@@ -1008,11 +1140,16 @@ define void @atomic_store_i32_seq_cst(ptr %a, i32 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i32_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sw a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i32_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sw a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i32_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sw a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i32 %b, ptr %a seq_cst, align 4
ret void
}
@@ -1126,11 +1263,16 @@ define void @atomic_store_i64_release(ptr %a, i64 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i64_release:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sd a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i64_release:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sd a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i64_release:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sd a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i64 %b, ptr %a release, align 8
ret void
}
@@ -1166,11 +1308,16 @@ define void @atomic_store_i64_seq_cst(ptr %a, i64 %b) nounwind {
; RV64I-NEXT: addi sp, sp, 16
; RV64I-NEXT: ret
;
-; RV64IA-LABEL: atomic_store_i64_seq_cst:
-; RV64IA: # %bb.0:
-; RV64IA-NEXT: fence rw, w
-; RV64IA-NEXT: sd a1, 0(a0)
-; RV64IA-NEXT: ret
+; RV64IA-WMO-LABEL: atomic_store_i64_seq_cst:
+; RV64IA-WMO: # %bb.0:
+; RV64IA-WMO-NEXT: fence rw, w
+; RV64IA-WMO-NEXT: sd a1, 0(a0)
+; RV64IA-WMO-NEXT: ret
+;
+; RV64IA-TSO-LABEL: atomic_store_i64_seq_cst:
+; RV64IA-TSO: # %bb.0:
+; RV64IA-TSO-NEXT: sd a1, 0(a0)
+; RV64IA-TSO-NEXT: ret
store atomic i64 %b, ptr %a seq_cst, align 8
ret void
}
More information about the llvm-commits
mailing list