[llvm] [IR] Add elementwise modifier to atomic stores (PR #210672)
via llvm-commits
llvm-commits at lists.llvm.org
Mon Jul 20 02:48:08 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-llvm-transforms
Author: Harrison Hao (harrisonGPU)
<details>
<summary>Changes</summary>
Add an elementwise modifier to atomic stores to represent
per-element atomic semantics for fixed-vector loads.
Without the modifier, a vector atomic stores remains a whole-value
atomic operation. With elementwise, the store behaves as if it were
expanded into one scalar atomic load per fixed-vector element, without
providing atomicity for the vector value as a whole.
Discussion: https://discourse.llvm.org/t/rfc-add-elementwise-modifier-to-atomic-loads-and-stores/91100
---
Patch is 26.46 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/210672.diff
15 Files Affected:
- (modified) llvm/docs/LangRef.md (+19-10)
- (modified) llvm/include/llvm/IR/Instructions.h (+11-2)
- (modified) llvm/lib/AsmParser/LLParser.cpp (+26-4)
- (modified) llvm/lib/Bitcode/Reader/BitcodeReader.cpp (+10-3)
- (modified) llvm/lib/Bitcode/Writer/BitcodeWriter.cpp (+12-8)
- (modified) llvm/lib/IR/AsmWriter.cpp (+3-1)
- (modified) llvm/lib/IR/Instruction.cpp (+1)
- (modified) llvm/lib/IR/Instructions.cpp (+5-3)
- (modified) llvm/lib/IR/Verifier.cpp (+21-4)
- (modified) llvm/lib/Transforms/Utils/FunctionComparator.cpp (+3)
- (modified) llvm/test/Assembler/atomic.ll (+4)
- (modified) llvm/test/Assembler/invalid-load-store-atomic-elementwise.ll (+56)
- (modified) llvm/test/Bitcode/atomic-load-store-elementwise.ll (+14)
- (modified) llvm/test/Bitcode/compatibility.ll (+6)
- (modified) llvm/unittests/IR/VerifierTest.cpp (+111)
``````````diff
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 9526cd020d724..e318c46669c8b 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -11858,7 +11858,7 @@ store i32 3, ptr %ptr ; yields void
```
store [volatile] <ty> <value>, ptr <pointer>[, align <alignment>][, !nontemporal !<nontemp_node>][, !invariant.group !<empty_node>] ; yields void
-store atomic [volatile] <ty> <value>, ptr <pointer> [syncscope("<target-scope>")] <ordering>, align <alignment> [, !invariant.group !<empty_node>] ; yields void
+store atomic [volatile] [elementwise] <ty> <value>, ptr <pointer> [syncscope("<target-scope>")] <ordering>, align <alignment> [, !invariant.group !<empty_node>] ; yields void
!<nontemp_node> = !{ i32 1 }
!<empty_node> = !{}
```
@@ -11876,16 +11876,25 @@ operand. If the `store` is marked as `volatile`, then the optimizer is not
allowed to modify the number or order of execution of this `store` with other
{ref}`volatile operations <volatile>`. Only values of {ref}`first class <t_firstclass>` types of known size (i.e., not containing an {ref}`opaque structural type <t_opaque>`) can be stored.
-If the `store` is marked as `atomic`, it takes an extra {ref}`ordering <ordering>` and optional `syncscope("<target-scope>")` argument. The
-`acquire` and `acq_rel` orderings aren't valid on `store` instructions.
-Atomic loads produce {ref}`defined <memmodel>` results when they may see
-multiple atomic stores. The type of the pointee must be an integer, pointer,
+If the `store` is marked as `atomic`, it takes an extra
+{ref}`ordering <ordering>` and optional `syncscope("<target-scope>")`
+argument. The `acquire` and `acq_rel` orderings are not valid on `store`
+instructions. The type of the stored value must be an integer, pointer,
floating-point, or vector type whose bit width is a power of two greater than
-or equal to eight. `align` must be
-explicitly specified on atomic stores. Note: if the alignment is not greater or
-equal to the size of the `<value>` type, the atomic operation is likely to
-require a lock and have poor performance. `!nontemporal` does not have any
-defined semantics for atomic stores.
+or equal to eight.
+
+If the `elementwise` modifier is present, the instruction has
+{ref}`elementwise atomic semantics <elementwise-atomics>`. The stored value
+must have a fixed vector type whose total bit width is a power of two greater
+than or equal to eight, and whose element type is supported by scalar atomic
+stores.
+
+`align` must be explicitly specified on atomic stores, and is otherwise
+optional on non-atomic stores. Note: if the alignment is not greater than or
+equal to the size of the `<value>` type, or the element type for an
+`elementwise` store, the atomic operation is likely to require a lock and have
+poor performance. `!nontemporal` does not have any defined semantics for
+atomic stores.
The optional constant `align` argument specifies the alignment of the
operation (that is, the alignment of the memory address). It is the
diff --git a/llvm/include/llvm/IR/Instructions.h b/llvm/include/llvm/IR/Instructions.h
index 5838109847845..84f581f5215c6 100644
--- a/llvm/include/llvm/IR/Instructions.h
+++ b/llvm/include/llvm/IR/Instructions.h
@@ -332,8 +332,9 @@ class StoreInst : public Instruction {
using VolatileField = BoolBitfieldElementT<0>;
using AlignmentField = AlignmentBitfieldElementT<VolatileField::NextBit>;
using OrderingField = AtomicOrderingBitfieldElementT<AlignmentField::NextBit>;
+ using ElementWiseField = BoolBitfieldElementT<OrderingField::NextBit>;
static_assert(
- Bitfield::areContiguous<VolatileField, AlignmentField, OrderingField>(),
+ Bitfield::areContiguous<VolatileField, AlignmentField, OrderingField, ElementWiseField>(),
"Bitfields must be contiguous");
void AssertOK();
@@ -370,6 +371,12 @@ class StoreInst : public Instruction {
/// Specify whether this is a volatile store or not.
void setVolatile(bool V) { setSubclassData<VolatileField>(V); }
+ /// Return true if this is an elementwise atomic store.
+ bool isElementwise() const { return getSubclassData<ElementWiseField>(); }
+
+ /// Specify whether this is an elementwise atomic store or not.
+ void setElementwise(bool V) { setSubclassData<ElementWiseField>(V); }
+
/// Transparently provide more efficient getOperand methods.
DECLARE_TRANSPARENT_OPERAND_ACCESSORS(Value);
@@ -412,7 +419,8 @@ class StoreInst : public Instruction {
/// Returns the properties of this store instruction.
LoadStoreInstProperties getProperties() const {
- return {isVolatile(), getAlign(), getOrdering(), getSyncScopeID()};
+ return {isVolatile(), getAlign(), getOrdering(), getSyncScopeID(),
+ isElementwise()};
}
/// Sets the properties of this store instruction.
@@ -421,6 +429,7 @@ class StoreInst : public Instruction {
setAlignment(Props.Alignment);
setOrdering(Props.Ordering);
setSyncScopeID(Props.SSID);
+ setElementwise(Props.IsElementwise);
}
bool isSimple() const { return !isAtomic() && !isVolatile(); }
diff --git a/llvm/lib/AsmParser/LLParser.cpp b/llvm/lib/AsmParser/LLParser.cpp
index c58b109b5ff9b..0b08cfea71d79 100644
--- a/llvm/lib/AsmParser/LLParser.cpp
+++ b/llvm/lib/AsmParser/LLParser.cpp
@@ -9030,10 +9030,11 @@ int LLParser::parseLoad(Instruction *&Inst, PerFunctionState &PFS) {
/// parseStore
/// ::= 'store' 'volatile'? TypeAndValue ',' TypeAndValue (',' 'align' i32)?
-/// ::= 'store' 'atomic' 'volatile'? TypeAndValue ',' TypeAndValue
-/// 'singlethread'? AtomicOrdering (',' 'align' i32)?
+/// ::= 'store' 'atomic' 'volatile'? 'elementwise'? TypeAndValue ','
+/// TypeAndValue 'singlethread'? AtomicOrdering (',' 'align' i32)?
int LLParser::parseStore(Instruction *&Inst, PerFunctionState &PFS) {
- Value *Val, *Ptr; LocTy Loc, PtrLoc;
+ Value *Val, *Ptr;
+ LocTy Loc, PtrLoc;
MaybeAlign Alignment;
bool AteExtraComma = false;
bool isAtomic = false;
@@ -9051,6 +9052,12 @@ int LLParser::parseStore(Instruction *&Inst, PerFunctionState &PFS) {
Lex.Lex();
}
+ bool IsElementwise = false;
+ if (Lex.getKind() == lltok::kw_elementwise) {
+ IsElementwise = true;
+ Lex.Lex();
+ }
+
if (parseTypeAndValue(Val, Loc, PFS) ||
parseToken(lltok::comma, "expected ',' after store operand") ||
parseTypeAndValue(Ptr, PtrLoc, PFS) ||
@@ -9067,13 +9074,28 @@ int LLParser::parseStore(Instruction *&Inst, PerFunctionState &PFS) {
if (Ordering == AtomicOrdering::Acquire ||
Ordering == AtomicOrdering::AcquireRelease)
return error(Loc, "atomic store cannot use Acquire ordering");
+
+ if (IsElementwise && !isAtomic)
+ return error(Loc, "elementwise store must be atomic");
+
+ if (IsElementwise && !isa<FixedVectorType>(Val->getType()))
+ return error(
+ Loc, "atomic elementwise store operand must have fixed vector type");
+
+ if (IsElementwise && Ordering == AtomicOrdering::SequentiallyConsistent)
+ return error(Loc,
+ "atomic elementwise store cannot be sequentially consistent");
+
SmallPtrSet<Type *, 4> Visited;
if (!Alignment && !Val->getType()->isSized(&Visited))
return error(Loc, "storing unsized types is not allowed");
if (!Alignment)
Alignment = M->getDataLayout().getABITypeAlign(Val->getType());
- Inst = new StoreInst(Val, Ptr, isVolatile, *Alignment, Ordering, SSID);
+ Inst = new StoreInst(Val, Ptr,
+ LoadStoreInstProperties{isVolatile, *Alignment, Ordering,
+ SSID, IsElementwise},
+ /*InsertBefore=*/nullptr);
return AteExtraComma ? InstExtraComma : InstNormal;
}
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index ac61ede6395af..fad22a72f7d71 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -6541,7 +6541,7 @@ Error BitcodeReader::parseFunctionBody(Function *F) {
}
case bitc::FUNC_CODE_INST_STOREATOMIC:
case bitc::FUNC_CODE_INST_STOREATOMIC_OLD: {
- // STOREATOMIC: [ptrty, ptr, val, align, vol, ordering, ssid]
+ // STOREATOMIC: [ptrty, ptr, val, align, vol, ordering, ssid, elementwise?]
unsigned OpNum = 0;
Value *Val, *Ptr;
unsigned PtrTypeID, ValTypeID;
@@ -6558,7 +6558,7 @@ Error BitcodeReader::parseFunctionBody(Function *F) {
return error("Invalid store atomic record");
}
- if (OpNum + 4 != Record.size())
+ if (OpNum + 4 != Record.size() && OpNum + 5 != Record.size())
return error("Invalid store atomic record");
if (Error Err = typeCheckLoadStoreInst(Val->getType(), Ptr->getType()))
@@ -6577,7 +6577,14 @@ Error BitcodeReader::parseFunctionBody(Function *F) {
return Err;
if (!Align)
return error("Alignment missing from atomic store");
- I = new StoreInst(Val, Ptr, Record[OpNum + 1], *Align, Ordering, SSID);
+
+ bool IsElementwise = Record.size() > OpNum + 4 && Record[OpNum + 4];
+
+ I = new StoreInst(
+ Val, Ptr,
+ LoadStoreInstProperties{/*IsVolatile=*/Record[OpNum + 1] != 0, *Align,
+ Ordering, SSID, IsElementwise},
+ /*InsertBefore=*/nullptr);
InstructionList.push_back(I);
break;
}
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index 571336c217797..9fd39e2f901ec 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -3590,8 +3590,9 @@ void ModuleBitcodeWriter::writeInstruction(const Instruction &I,
break;
}
- case Instruction::Store:
- if (cast<StoreInst>(I).isAtomic()) {
+ case Instruction::Store: {
+ const auto &SI = cast<StoreInst>(I);
+ if (SI.isAtomic()) {
Code = bitc::FUNC_CODE_INST_STOREATOMIC;
} else {
Code = bitc::FUNC_CODE_INST_STORE;
@@ -3601,14 +3602,17 @@ void ModuleBitcodeWriter::writeInstruction(const Instruction &I,
AbbrevToUse = 0;
if (pushValueAndType(I.getOperand(0), InstID, Vals)) // valty + val
AbbrevToUse = 0;
- Vals.push_back(getEncodedAlign(cast<StoreInst>(I).getAlign()));
- Vals.push_back(cast<StoreInst>(I).isVolatile());
- if (cast<StoreInst>(I).isAtomic()) {
- Vals.push_back(getEncodedOrdering(cast<StoreInst>(I).getOrdering()));
- Vals.push_back(
- getEncodedSyncScopeID(cast<StoreInst>(I).getSyncScopeID()));
+ Vals.push_back(getEncodedAlign(SI.getAlign()));
+ Vals.push_back(SI.isVolatile());
+ if (SI.isAtomic()) {
+ Vals.push_back(getEncodedOrdering(SI.getOrdering()));
+ Vals.push_back(getEncodedSyncScopeID(SI.getSyncScopeID()));
+ if (SI.isElementwise())
+ Vals.push_back(1);
}
break;
+ }
+
case Instruction::AtomicCmpXchg:
Code = bitc::FUNC_CODE_INST_CMPXCHG;
pushValueAndType(I.getOperand(0), InstID, Vals); // ptrty + ptr
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index cad4f17b0db91..2555da6cfd87f 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -4480,7 +4480,9 @@ void AssemblyWriter::printInstruction(const Instruction &I) {
(isa<AtomicRMWInst>(I) && cast<AtomicRMWInst>(I).isVolatile()))
Out << " volatile";
- if (isa<LoadInst>(I) && cast<LoadInst>(I).isElementwise())
+ // Print the elementwise marker for atomic loads and stores.
+ if ((isa<LoadInst>(I) && cast<LoadInst>(I).isElementwise()) ||
+ (isa<StoreInst>(I) && cast<StoreInst>(I).isElementwise()))
Out << " elementwise";
// Print out optimization information.
diff --git a/llvm/lib/IR/Instruction.cpp b/llvm/lib/IR/Instruction.cpp
index 5bade74b93c9b..ece1447696959 100644
--- a/llvm/lib/IR/Instruction.cpp
+++ b/llvm/lib/IR/Instruction.cpp
@@ -934,6 +934,7 @@ bool Instruction::hasSameSpecialState(const Instruction *I2,
LI->getSyncScopeID() == cast<LoadInst>(I2)->getSyncScopeID();
if (const StoreInst *SI = dyn_cast<StoreInst>(I1))
return SI->isVolatile() == cast<StoreInst>(I2)->isVolatile() &&
+ SI->isElementwise() == cast<StoreInst>(I2)->isElementwise() &&
(SI->getAlign() == cast<StoreInst>(I2)->getAlign() ||
IgnoreAlignment) &&
SI->getOrdering() == cast<StoreInst>(I2)->getOrdering() &&
diff --git a/llvm/lib/IR/Instructions.cpp b/llvm/lib/IR/Instructions.cpp
index e40aae61c723c..1c7e671448fc6 100644
--- a/llvm/lib/IR/Instructions.cpp
+++ b/llvm/lib/IR/Instructions.cpp
@@ -1412,7 +1412,9 @@ StoreInst::StoreInst(Value *Val, Value *Ptr,
const LoadStoreInstProperties &Props,
InsertPosition InsertBefore)
: StoreInst(Val, Ptr, Props.IsVolatile, Props.Alignment, Props.Ordering,
- Props.SSID, InsertBefore) {}
+ Props.SSID, InsertBefore) {
+ setElementwise(Props.IsElementwise);
+}
StoreInst::StoreInst(Value *val, Value *addr, bool isVolatile, Align Align,
AtomicOrdering Order, SyncScope::ID SSID,
@@ -4460,8 +4462,8 @@ LoadInst *LoadInst::cloneImpl() const {
}
StoreInst *StoreInst::cloneImpl() const {
- return new StoreInst(getOperand(0), getOperand(1), isVolatile(), getAlign(),
- getOrdering(), getSyncScopeID());
+ return new StoreInst(getOperand(0), getOperand(1), getProperties(),
+ /*InsertBefore=*/nullptr);
}
AtomicCmpXchgInst *AtomicCmpXchgInst::cloneImpl() const {
diff --git a/llvm/lib/IR/Verifier.cpp b/llvm/lib/IR/Verifier.cpp
index 28459a39fedae..4b3f5e4c51049 100644
--- a/llvm/lib/IR/Verifier.cpp
+++ b/llvm/lib/IR/Verifier.cpp
@@ -4630,14 +4630,31 @@ void Verifier::visitStoreInst(StoreInst &SI) {
Check(SI.getOrdering() != AtomicOrdering::Acquire &&
SI.getOrdering() != AtomicOrdering::AcquireRelease,
"Store cannot have Acquire ordering", &SI);
- Check(ElTy->getScalarType()->isIntOrPtrTy() ||
- ElTy->getScalarType()->isByteTy() ||
- ElTy->getScalarType()->isFloatingPointTy(),
+
+ Type *ScalarTy = ElTy;
+ if (SI.isElementwise()) {
+ Check(SI.getOrdering() != AtomicOrdering::SequentiallyConsistent,
+ "atomic elementwise store cannot be sequentially consistent.", &SI);
+
+ auto *VecTy = dyn_cast<FixedVectorType>(ElTy);
+ Check(VecTy,
+ "atomic elementwise store operand must have fixed vector type!",
+ &SI, ElTy);
+ if (VecTy) {
+ checkAtomicMemAccessSize(ScalarTy, &SI);
+ ScalarTy = VecTy->getElementType();
+ }
+ }
+
+ Check(ScalarTy->getScalarType()->isIntOrPtrTy() ||
+ ScalarTy->getScalarType()->isByteTy() ||
+ ScalarTy->getScalarType()->isFloatingPointTy(),
"atomic store operand must have integer, byte, pointer, floating "
"point, or vector type!",
ElTy, &SI);
- checkAtomicMemAccessSize(ElTy, &SI);
+ checkAtomicMemAccessSize(ScalarTy, &SI);
} else {
+ Check(!SI.isElementwise(), "non-atomic store cannot be elementwise", &SI);
Check(SI.getSyncScopeID() == SyncScope::System,
"Non-atomic store cannot have SynchronizationScope specified", &SI);
}
diff --git a/llvm/lib/Transforms/Utils/FunctionComparator.cpp b/llvm/lib/Transforms/Utils/FunctionComparator.cpp
index 80b02d579f5da..3cdb05f15f367 100644
--- a/llvm/lib/Transforms/Utils/FunctionComparator.cpp
+++ b/llvm/lib/Transforms/Utils/FunctionComparator.cpp
@@ -713,6 +713,9 @@ int FunctionComparator::cmpOperations(const Instruction *L,
if (int Res =
cmpNumbers(SI->isVolatile(), cast<StoreInst>(R)->isVolatile()))
return Res;
+ if (int Res = cmpNumbers(SI->isElementwise(),
+ cast<StoreInst>(R)->isElementwise()))
+ return Res;
if (int Res = cmpAligns(SI->getAlign(), cast<StoreInst>(R)->getAlign()))
return Res;
if (int Res =
diff --git a/llvm/test/Assembler/atomic.ll b/llvm/test/Assembler/atomic.ll
index afe4f48d3ac30..30f4011937b1b 100644
--- a/llvm/test/Assembler/atomic.ll
+++ b/llvm/test/Assembler/atomic.ll
@@ -75,6 +75,10 @@ define void @f(ptr %x) {
load atomic elementwise <2 x float>, ptr %x syncscope("agent") monotonic, align 4
; CHECK: load atomic volatile elementwise <2 x i32>, ptr %x monotonic, align 4
load atomic volatile elementwise <2 x i32>, ptr %x monotonic, align 4
+ ; CHECK: store atomic elementwise <2 x float> <float 3.000000e+00, float 4.000000e+00>, ptr %x syncscope("agent") monotonic, align 4
+ store atomic elementwise <2 x float> <float 3.0, float 4.0>, ptr %x syncscope("agent") monotonic, align 4
+ ; CHECK: store atomic volatile elementwise <2 x i32> <i32 3, i32 4>, ptr %x monotonic, align 4
+ store atomic volatile elementwise <2 x i32> <i32 3, i32 4>, ptr %x monotonic, align 4
; CHECK: fence syncscope("singlethread") release
fence syncscope("singlethread") release
diff --git a/llvm/test/Assembler/invalid-load-store-atomic-elementwise.ll b/llvm/test/Assembler/invalid-load-store-atomic-elementwise.ll
index 843564cbbc075..d8d3a7885306b 100644
--- a/llvm/test/Assembler/invalid-load-store-atomic-elementwise.ll
+++ b/llvm/test/Assembler/invalid-load-store-atomic-elementwise.ll
@@ -5,6 +5,13 @@
; RUN: not llvm-as -disable-output %t/load-odd-sized.ll 2>&1 | FileCheck %t/load-odd-sized.ll
; RUN: not llvm-as -disable-output %t/load-non-byte.ll 2>&1 | FileCheck %t/load-non-byte.ll
; RUN: not llvm-as -disable-output %t/load-non-byte-element.ll 2>&1 | FileCheck %t/load-non-byte-element.ll
+; RUN: not llvm-as -disable-output %t/store-non-atomic.ll 2>&1 | FileCheck %t/store-non-atomic.ll
+; RUN: not llvm-as -disable-output %t/store-scalar.ll 2>&1 | FileCheck %t/store-scalar.ll
+; RUN: not llvm-as -disable-output %t/store-scalable.ll 2>&1 | FileCheck %t/store-scalable.ll
+; RUN: not llvm-as -disable-output %t/store-odd-sized.ll 2>&1 | FileCheck %t/store-odd-sized.ll
+; RUN: not llvm-as -disable-output %t/store-non-byte.ll 2>& 1 | FileCheck %t/store-non-byte.ll
+; RUN: not llvm-as -disable-output %t/store-non-byte-element.ll 2>&1 | FileCheck %t/store-non-byte-element.ll
+; RUN: not llvm-as -disable-output %t/store-seq-cst.ll 2>&1 | FileCheck %t/store-seq-cst.ll
;--- load-non-atomic.ll
; CHECK: elementwise load must be atomic
@@ -47,3 +54,52 @@ define <8 x i1> @bad_non_byte_element(ptr %p) {
%v = load atomic elementwise <8 x i1>, ptr %p monotonic, align 1
ret <8 x i1> %v
}
+
+;--- store-non-atomic.ll
+; CHECK: elementwise store must be atomic
+define void @bad_non_atomic_store(ptr %p, <2 x float> %v) {
+ store elementwise <2 x float> %v, ptr %p, align 4
+ ret void
+}
+
+;--- store-scalar.ll
+; CHECK: atomic elementwise store operand must have fixed vector type
+define void @bad_scalar_store(ptr %p, float %v) {
+ store atomic elementwise float %v, ptr %p monotonic, align 4
+ ret void
+}
+
+;--- store-scalable.ll
+; CHECK: atomic elementwise store operand must have fixed vector type
+define void @bad_scalable_store(ptr %p, <vscale x 2 x i32> %v) {
+ store atomic elementwise <vscale x 2 x i32> %v, ptr %p monotonic, align 4
+ ret void
+}
+
+;--- store-odd-sized.ll
+; CHECK: atomic memory access' operand must have a power-of-two size
+define void @bad_odd_sized_vector_store(ptr %p, <5 x i32> %v) {
+ store atomic elementwise <5 x i32> %v, ptr %p monotonic, align 4
+ ret void
+}
+
+;--- store-non-byte.ll
+; CHECK: atomic memory access' size must be byte-sized
+define void @bad_non_byte_store(ptr %p, <4 x i1> %v) {
+ store atomic elementwise <4 x i1> %v, ptr %p monotonic, align 4
+ ret void
+}
+
+;--- store-non-byte-element.ll
+; CHECK: atomic memory access' size must be byte-sized
+define void @bad_non_byte_element_store(ptr %p, <8 x i1> %v) {
+ store atomic elementwise <8 x i1> %v, ptr %p monotonic, align 1
+ ret void
+}
+
+;--- store-seq-cst.ll
+; CHECK: atomic elementwise store cannot be sequentially consistent
+define void @bad_store_seq_cst(ptr %p, <4 x i32> %v) {
+ store atomic elementwise <4 x i32> %v, ptr %p seq_cst, align 4
+ ret void
+}
diff --git a/llvm/test/Bitcode/atomic-load-store-elementwise.ll b/llvm/test/Bitcode/atomic-load-store-elementwise.ll
index 6b921c199c08c..c26708a90beb2 100644
--- a/llvm/test/Bitcode/atomic-load-store-elementwise.ll
+++ b/llvm/test/Bitcode/atomic-load-store-ele...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/210672
More information about the llvm-commits
mailing list