[llvm] f3fb973 - [AArch64] Comprehensive tests for atomic operations

[Tomas Matheson via llvm-commits]

Author: Tomas Matheson
Date: 2023-01-10T14:02:13Z
New Revision: f3fb973924a9ba3bdd8dfee4dc85337e8b033ab4

URL: https://github.com/llvm/llvm-project/commit/f3fb973924a9ba3bdd8dfee4dc85337e8b033ab4
DIFF: https://github.com/llvm/llvm-project/commit/f3fb973924a9ba3bdd8dfee4dc85337e8b033ab4.diff

LOG: [AArch64] Comprehensive tests for atomic operations

There are a lot of variants of atomic operations, and AArch64 has several
distinct options to codegen them, and does different things depending on
available features, architecture version and optimisation level. The current
testing for atomic operations has been added gradually over time and does not
give full coverate.  Given how complex the codegen for atomic operations is, it
is valuable to cover the entire parameter space, i.e. test them all. The
resulting set of tests serve also as a reference for how each is codegened.

In order to keep the test files readable and avoid constant updating for
unrelated codegen changes, the test outputs are filtered to only include the
relevant instructions. This shows for each operation and feature which codegen
approach is taken (e.g. ll/sc loop, atomic instruction, library call).

The following parameter space is tested:
 - feature: +lse, +rcpc, etc
 - optimisation level: O0, O1 (covers GISel and SelectionDAG)
 - atomic instruction: load, store, cmpxchg, atomirmw*
 - size: i8, i16, i32, i64, i128
 - aligned/unaligned accesses
 - endianness: big, little
 - atomic ordering: release, acquire, etc
 - load atomic only: const/non-const
 - cmpxchg only: weak/strong
 - atomicrmw: update operation (add, sub, etc)

Notably not covered:
 - volatility: there should be no difference between volatile/non-volatile
 - atomicrmw fadd/fsub

The files are split by triple, LLVM instruction, and feature. This makes it
easy to diff between features and endianness for a given instruction.

The file that generates the tests is included.

There are 70 test files with an average of 2800 lines each.

Differential Revision: https://reviews.llvm.org/D141064

Added: 
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64-fence.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse128.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse2.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-outline_atomics.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc3.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8_1a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8a.ll
    llvm/test/CodeGen/AArch64/Atomics/aarch64_be-fence.ll
    llvm/test/CodeGen/AArch64/Atomics/generate-tests.py

Modified: 
    

Removed: 
    


################################################################################
diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse128.ll
new file mode 100644
index 000000000000..b028a5cfba3e
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse128.ll
@@ -0,0 +1,588 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    caspa x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    caspal x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse2.ll
new file mode 100644
index 000000000000..e7e231bc344d
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-lse2.ll
@@ -0,0 +1,571 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered:
+; CHECK:    ldp x0, x1, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered_const:
+; CHECK:    ldp x0, x1, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic:
+; CHECK:    ldp x0, x1, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic_const:
+; CHECK:    ldp x0, x1, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire:
+; CHECK:    ldp x0, x1, [x0]
+; CHECK:    dmb ishld
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire_const:
+; CHECK:    ldp x0, x1, [x0]
+; CHECK:    dmb ishld
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst:
+; CHECK:    ldp x0, x1, [x0]
+; CHECK:    dmb ish
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; CHECK:    ldp x0, x1, [x0]
+; CHECK:    dmb ish
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-outline_atomics.ll
new file mode 100644
index 000000000000..fb4bef33d9b4
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-outline_atomics.ll
@@ -0,0 +1,628 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc.ll
new file mode 100644
index 000000000000..373b040ebec6
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc.ll
@@ -0,0 +1,628 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc3.ll
new file mode 100644
index 000000000000..df73e52c0c1d
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-rcpc3.ll
@@ -0,0 +1,628 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8_1a.ll
new file mode 100644
index 000000000000..e14610ba6b1c
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8_1a.ll
@@ -0,0 +1,588 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    casp x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    caspa x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    caspal x2, x3, x0, x1, [x8]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8a.ll
new file mode 100644
index 000000000000..045e080983d5
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-load-v8a.ll
@@ -0,0 +1,628 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    ldxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    ldaxp x0, x1, [x9]
+; -O0:    cmp x0, x10
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x0, x1, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    stlxp w9, x0, x1, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse128.ll
new file mode 100644
index 000000000000..e8437d86ae33
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse128.ll
@@ -0,0 +1,324 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspl x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspal x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse2.ll
new file mode 100644
index 000000000000..e90e5fb2d95a
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-lse2.ll
@@ -0,0 +1,290 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_unordered:
+; CHECK:    stp x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_monotonic:
+; CHECK:    stp x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_release:
+; CHECK:    dmb ish
+; CHECK:    stp x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_seq_cst:
+; CHECK:    dmb ish
+; CHECK:    stp x0, x1, [x2]
+; CHECK:    dmb ish
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-outline_atomics.ll
new file mode 100644
index 000000000000..3d204b734d4a
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-outline_atomics.ll
@@ -0,0 +1,332 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc.ll
new file mode 100644
index 000000000000..c8971b1b877b
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc.ll
@@ -0,0 +1,332 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc3.ll
new file mode 100644
index 000000000000..cf7d512548b1
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-rcpc3.ll
@@ -0,0 +1,332 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8_1a.ll
new file mode 100644
index 000000000000..00bcedf4b7a6
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8_1a.ll
@@ -0,0 +1,324 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspl x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspal x6, x7, x0, x1, [x2]
+; -O1:    cmp x7, x5
+; -O1:    ccmp x6, x4, #0, eq
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8a.ll
new file mode 100644
index 000000000000..250ae1b99e88
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomic-store-v8a.ll
@@ -0,0 +1,332 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x0, x1, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse128.ll
new file mode 100644
index 000000000000..27aedd35c5d0
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse128.ll
@@ -0,0 +1,6734 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; CHECK:    swph w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; CHECK:    swpah w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_release:
+; CHECK:    swplh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; CHECK:    swp w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; CHECK:    swpa w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_release:
+; CHECK:    swpl w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; CHECK:    swp x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; CHECK:    swpa x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_release:
+; CHECK:    swpl x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    casp x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspa x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspl x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; CHECK:    ldaddh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acquire:
+; CHECK:    ldaddah w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_release:
+; CHECK:    ldaddlh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; CHECK:    ldadd w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acquire:
+; CHECK:    ldadda w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_release:
+; CHECK:    ldaddl w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; CHECK:    ldadd x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acquire:
+; CHECK:    ldadda x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_release:
+; CHECK:    ldaddl x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x2, x10, x12
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x2, x10, x12
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x2, x10, x12
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; CHECK:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; CHECK:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_release:
+; CHECK:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; CHECK:    ldaddh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; CHECK:    ldaddah w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_release:
+; CHECK:    ldaddlh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; CHECK:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; CHECK:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; CHECK:    ldadd w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; CHECK:    ldadda w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_release:
+; CHECK:    ldaddl w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; CHECK:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; CHECK:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; CHECK:    ldadd x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; CHECK:    ldadda x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_release:
+; CHECK:    ldaddl x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; CHECK:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; CHECK:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x2, x10, x12
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x2, x10, x12
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x2, x10, x12
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; CHECK:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; CHECK:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_release:
+; CHECK:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrah w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclr w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclra w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrl w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclr x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acquire:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclra x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_release:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclrl x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casab w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    caslb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    cash w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cash w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casah w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casah w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    caslh w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casalh w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casalh w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    cas w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cas w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casa w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casa w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casl w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casl w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casal w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casal w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    cas x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    cas x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casa x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casa x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casl x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casl x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casal x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casal x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    casp x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspa x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspl x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casab w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    caslb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; CHECK:    ldseth w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acquire:
+; CHECK:    ldsetah w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_release:
+; CHECK:    ldsetlh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; CHECK:    ldset w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acquire:
+; CHECK:    ldseta w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_release:
+; CHECK:    ldsetl w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; CHECK:    ldset x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acquire:
+; CHECK:    ldseta x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_release:
+; CHECK:    ldsetl x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; CHECK:    ldeorh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; CHECK:    ldeorah w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_release:
+; CHECK:    ldeorlh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; CHECK:    ldeor w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; CHECK:    ldeora w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_release:
+; CHECK:    ldeorl w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; CHECK:    ldeor x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; CHECK:    ldeora x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_release:
+; CHECK:    ldeorl x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; CHECK:    ldsmaxh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acquire:
+; CHECK:    ldsmaxah w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_release:
+; CHECK:    ldsmaxlh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; CHECK:    ldsmax w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acquire:
+; CHECK:    ldsmaxa w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_release:
+; CHECK:    ldsmaxl w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; CHECK:    ldsmax x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acquire:
+; CHECK:    ldsmaxa x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_release:
+; CHECK:    ldsmaxl x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; CHECK:    ldsminh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acquire:
+; CHECK:    ldsminah w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_release:
+; CHECK:    ldsminlh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; CHECK:    ldsmin w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acquire:
+; CHECK:    ldsmina w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_release:
+; CHECK:    ldsminl w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; CHECK:    ldsmin x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acquire:
+; CHECK:    ldsmina x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_release:
+; CHECK:    ldsminl x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; CHECK:    ldumaxh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; CHECK:    ldumaxah w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_release:
+; CHECK:    ldumaxlh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; CHECK:    ldumax w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; CHECK:    ldumaxa w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_release:
+; CHECK:    ldumaxl w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; CHECK:    ldumax x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; CHECK:    ldumaxa x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_release:
+; CHECK:    ldumaxl x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; CHECK:    lduminh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; CHECK:    lduminah w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_release:
+; CHECK:    lduminlh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; CHECK:    ldumin w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; CHECK:    ldumina w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_release:
+; CHECK:    lduminl w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; CHECK:    ldumin x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; CHECK:    ldumina x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_release:
+; CHECK:    lduminl x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2.ll
new file mode 100644
index 000000000000..f94af34076e7
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2.ll
@@ -0,0 +1,9679 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-outline_atomics.ll
new file mode 100644
index 000000000000..16bf4c269088
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-outline_atomics.ll
@@ -0,0 +1,8504 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_swp1_relax
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; CHECK:    bl __aarch64_swp1_acq
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_release:
+; CHECK:    bl __aarch64_swp1_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_swp2_relax
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; CHECK:    bl __aarch64_swp2_acq
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_release:
+; CHECK:    bl __aarch64_swp2_rel
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp2_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp2_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_swp4_relax
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; CHECK:    bl __aarch64_swp4_acq
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_release:
+; CHECK:    bl __aarch64_swp4_rel
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp4_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp4_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_swp8_relax
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; CHECK:    bl __aarch64_swp8_acq
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_release:
+; CHECK:    bl __aarch64_swp8_rel
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp8_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp8_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_swp1_relax
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_swp1_acq
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; CHECK:    bl __aarch64_swp1_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_release:
+; CHECK:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd2_relax
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd2_acq
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_release:
+; CHECK:    bl __aarch64_ldadd2_rel
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd4_relax
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd4_acq
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_release:
+; CHECK:    bl __aarch64_ldadd4_rel
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd8_relax
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd8_acq
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_release:
+; CHECK:    bl __aarch64_ldadd8_rel
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x2, x0, x9
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x2, x0, x9
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x2, x0, x9
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x2, x0, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x2, x0, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_release:
+; CHECK:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_relax
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_relax
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd2_relax
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_acq
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    bl __aarch64_ldadd2_acq
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_rel
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    bl __aarch64_ldadd2_rel
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_relax
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd4_relax
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_acq
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    bl __aarch64_ldadd4_acq
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_rel
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    bl __aarch64_ldadd4_rel
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_relax
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd8_relax
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_acq
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    bl __aarch64_ldadd8_acq
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_rel
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    bl __aarch64_ldadd8_rel
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x2, x0, x9
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x2, x0, x9
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x2, x0, x9
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x2, x0, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x2, x0, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_relax
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_relax
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_relax
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_rel
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_relax
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_relax
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_acq
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_acq
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_rel
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_rel
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_acq_rel
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_acq_rel
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_relax
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_relax
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_acq
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_acq
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_rel
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_rel
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_acq_rel
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_acq_rel
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_relax
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_relax
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_acq
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_acq
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_rel
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_rel
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_acq_rel
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_acq_rel
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_relax
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_relax
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_rel
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset1_relax
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acquire:
+; CHECK:    bl __aarch64_ldset1_acq
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_release:
+; CHECK:    bl __aarch64_ldset1_rel
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset2_relax
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acquire:
+; CHECK:    bl __aarch64_ldset2_acq
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_release:
+; CHECK:    bl __aarch64_ldset2_rel
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset2_acq_rel
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset2_acq_rel
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset4_relax
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acquire:
+; CHECK:    bl __aarch64_ldset4_acq
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_release:
+; CHECK:    bl __aarch64_ldset4_rel
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset4_acq_rel
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset4_acq_rel
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset8_relax
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acquire:
+; CHECK:    bl __aarch64_ldset8_acq
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_release:
+; CHECK:    bl __aarch64_ldset8_rel
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset8_acq_rel
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset8_acq_rel
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_ldset1_relax
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_ldset1_acq
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_release:
+; CHECK:    bl __aarch64_ldset1_rel
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor1_relax
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor1_acq
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_release:
+; CHECK:    bl __aarch64_ldeor1_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor2_relax
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor2_acq
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_release:
+; CHECK:    bl __aarch64_ldeor2_rel
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor2_acq_rel
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor2_acq_rel
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor4_relax
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor4_acq
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_release:
+; CHECK:    bl __aarch64_ldeor4_rel
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor4_acq_rel
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor4_acq_rel
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor8_relax
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor8_acq
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_release:
+; CHECK:    bl __aarch64_ldeor8_rel
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor8_acq_rel
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor8_acq_rel
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_ldeor1_relax
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_ldeor1_acq
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_release:
+; CHECK:    bl __aarch64_ldeor1_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc.ll
new file mode 100644
index 000000000000..3a150acbf5aa
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc.ll
@@ -0,0 +1,9679 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc3.ll
new file mode 100644
index 000000000000..838f4529c4dc
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc3.ll
@@ -0,0 +1,9679 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8_1a.ll
new file mode 100644
index 000000000000..40a055e50590
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8_1a.ll
@@ -0,0 +1,6734 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; CHECK:    swph w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; CHECK:    swpah w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_release:
+; CHECK:    swplh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; CHECK:    swp w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; CHECK:    swpa w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_release:
+; CHECK:    swpl w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; CHECK:    swp x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; CHECK:    swpa x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_release:
+; CHECK:    swpl x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    casp x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspa x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspl x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; CHECK:    ldaddh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acquire:
+; CHECK:    ldaddah w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_release:
+; CHECK:    ldaddlh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; CHECK:    ldadd w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acquire:
+; CHECK:    ldadda w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_release:
+; CHECK:    ldaddl w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; CHECK:    ldadd x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acquire:
+; CHECK:    ldadda x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_release:
+; CHECK:    ldaddl x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x2, x10, x12
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x2, x10, x12
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x2, x10, x12
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; CHECK:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; CHECK:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_release:
+; CHECK:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; CHECK:    ldaddh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; CHECK:    ldaddah w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_release:
+; CHECK:    ldaddlh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; CHECK:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; CHECK:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; CHECK:    ldadd w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; CHECK:    ldadda w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_release:
+; CHECK:    ldaddl w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; CHECK:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; CHECK:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; CHECK:    ldadd x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; CHECK:    ldadda x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_release:
+; CHECK:    ldaddl x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; CHECK:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; CHECK:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x2, x10, x12
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x2, x10, x12
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x2, x10, x12
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x2, x10, x12
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x8, x11
+; -O0:    ccmp x9, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x8, x4, x2
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; CHECK:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; CHECK:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_release:
+; CHECK:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; CHECK:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrah w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclr w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclra w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrl w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclr x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acquire:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclra x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_release:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclrl x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x2, x9, x11
+; -O0:    and x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casab w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    caslb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    cash w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cash w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casah w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casah w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    caslh w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casalh w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casalh w9, w10, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    cas w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cas w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casa w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casa w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casl w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casl w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casal w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w10, w9
+; -O0:    casal w9, w10, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    cas x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    cas x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casa x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casa x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casl x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casl x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casal x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x10, x9
+; -O0:    casal x9, x10, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    casp x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspa x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspl x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x10, x9, x10
+; -O0:    and x9, x9, x11
+; -O0:    mvn x2, x10
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casab w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    caslb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w8, w8
+; -O0:    casalb w9, w8, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; CHECK:    ldseth w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acquire:
+; CHECK:    ldsetah w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_release:
+; CHECK:    ldsetlh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; CHECK:    ldset w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acquire:
+; CHECK:    ldseta w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_release:
+; CHECK:    ldsetl w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; CHECK:    ldset x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acquire:
+; CHECK:    ldseta x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_release:
+; CHECK:    ldsetl x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x2, x9, x11
+; -O0:    orr x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; CHECK:    ldeorh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; CHECK:    ldeorah w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_release:
+; CHECK:    ldeorlh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; CHECK:    ldeor w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; CHECK:    ldeora w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_release:
+; CHECK:    ldeorl w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; CHECK:    ldeor x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; CHECK:    ldeora x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_release:
+; CHECK:    ldeorl x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x2, x9, x11
+; -O0:    eor x9, x9, x10
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; CHECK:    ldsmaxh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acquire:
+; CHECK:    ldsmaxah w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_release:
+; CHECK:    ldsmaxlh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; CHECK:    ldsmax w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acquire:
+; CHECK:    ldsmaxa w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_release:
+; CHECK:    ldsmaxl w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; CHECK:    ldsmax x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acquire:
+; CHECK:    ldsmaxa x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_release:
+; CHECK:    ldsmaxl x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; CHECK:    ldsminh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acquire:
+; CHECK:    ldsminah w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_release:
+; CHECK:    ldsminlh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; CHECK:    ldsmin w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acquire:
+; CHECK:    ldsmina w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_release:
+; CHECK:    ldsminl w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; CHECK:    ldsmin x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acquire:
+; CHECK:    ldsmina x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_release:
+; CHECK:    ldsminl x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; CHECK:    ldumaxh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; CHECK:    ldumaxah w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_release:
+; CHECK:    ldumaxlh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; CHECK:    ldumax w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; CHECK:    ldumaxa w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_release:
+; CHECK:    ldumaxl w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; CHECK:    ldumax x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; CHECK:    ldumaxa x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_release:
+; CHECK:    ldumaxl x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; CHECK:    lduminh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; CHECK:    lduminah w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_release:
+; CHECK:    lduminlh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; CHECK:    ldumin w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; CHECK:    ldumina w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_release:
+; CHECK:    lduminl w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; CHECK:    ldumin x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; CHECK:    ldumina x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_release:
+; CHECK:    lduminl x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x10
+; -O0:    subs x9, x9, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w9, w9, w11, ne
+; -O0:    and w13, w9, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x2, x11, x12, ne
+; -O0:    and w11, w9, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x9, x9, x10, ne
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x2, x4
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x5, x7
+; -O1:    ccmp x4, x6, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8a.ll
new file mode 100644
index 000000000000..c9e0ff0f5d49
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8a.ll
@@ -0,0 +1,9679 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x8, x1, [x0]
+; -O1:    stlxp w9, x2, x3, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8, uxth
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    adds x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w8, w10, uxth
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w8, w9, uxth
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    subs x9, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x8, x11, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x14, x8, x10
+; -O0:    and x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    and x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w9, w8, w9
+; -O0:    mvn w12, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    mvn x12, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x14, x9
+; -O0:    mvn x15, x8
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    and x9, x0, x2
+; -O1:    mvn x9, x9
+; -O1:    and x10, x1, x3
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w10, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x8, x9
+; -O0:    and x8, x8, x10
+; -O0:    mvn x9, x9
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x14, x8, x10
+; -O0:    orr x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    orr x9, x0, x2
+; -O1:    orr x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x9, x8, x9
+; -O0:    orr x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w8, w9
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x8, x9
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x14, x8, x10
+; -O0:    eor x15, x8, x9
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    eor x9, x0, x2
+; -O1:    eor x10, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w10, w8
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x9, x8, x9
+; -O0:    eor x8, x8, x10
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lt
+; -O1:    csel x10, x0, x2, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w8
+; -O0:    subs w10, w10, w9, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, ge
+; -O1:    csel x10, x0, x2, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w10
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, lo
+; -O1:    csel x10, x0, x2, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    subs w10, w10, w9, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxrh w9, [x11]
+; -O0:    cmp w9, w8, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w8, w8, w9, uxth
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w8, w9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w12, w8, w9, ne
+; -O0:    ldaxr w9, [x11]
+; -O0:    cmp w9, w8
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w8, w9, w8
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x8, x9
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x12, x8, x9, ne
+; -O0:    ldaxr x9, [x11]
+; -O0:    cmp x9, x8
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x8, x9, x8
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stxp w8, x14, x15, [x11]
+; -O0:    stxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x9
+; -O0:    subs x8, x8, x12
+; -O0:    and w13, w13, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel w8, w8, w10, ne
+; -O0:    and w13, w8, #0x1
+; -O0:    ands w13, w13, #0x1
+; -O0:    csel x14, x10, x12, ne
+; -O0:    and w10, w8, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x15, x8, x9, ne
+; -O0:    ldaxp x10, x9, [x11]
+; -O0:    cmp x10, x12
+; -O0:    cmp x9, x13
+; -O0:    stlxp w8, x14, x15, [x11]
+; -O0:    stlxp w8, x10, x9, [x11]
+; -O0:    eor x8, x10, x8
+; -O0:    eor x11, x9, x11
+; -O0:    orr x8, x8, x11
+; -O0:    subs x8, x8, #0
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x0, x1, [x8]
+; -O1:    cmp x2, x0
+; -O1:    csel x9, x1, x3, hs
+; -O1:    csel x10, x0, x2, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w10, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    and w9, w9, #0x1
+; -O0:    ands w9, w9, #0x1
+; -O0:    csel w12, w10, w8, ne
+; -O0:    ldaxrb w9, [x11]
+; -O0:    cmp w9, w10, uxtb
+; -O0:    stlxrb w8, w12, [x11]
+; -O0:    and w8, w9, #0xff
+; -O0:    subs w8, w8, w10, uxtb
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel w8, w9, w8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    and w10, w10, #0x1
+; -O0:    ands w10, w10, #0x1
+; -O0:    csel x8, x9, x8, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x10
+; -O0:    subs x8, x8, x11
+; -O0:    and w12, w12, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel w8, w8, w9, ne
+; -O0:    and w12, w8, #0x1
+; -O0:    ands w12, w12, #0x1
+; -O0:    csel x9, x9, x11, ne
+; -O0:    and w11, w8, #0x1
+; -O0:    ands w11, w11, #0x1
+; -O0:    csel x8, x8, x10, ne
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x20, x0
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse128.ll
new file mode 100644
index 000000000000..5f842e000b0f
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse128.ll
@@ -0,0 +1,2494 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse2.ll
new file mode 100644
index 000000000000..2988242efe2d
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-lse2.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-outline_atomics.ll
new file mode 100644
index 000000000000..403e4770e17f
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-outline_atomics.ll
@@ -0,0 +1,3364 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    bl __aarch64_cas1_relax
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    bl __aarch64_cas1_relax
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    bl __aarch64_cas1_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    bl __aarch64_cas1_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    bl __aarch64_cas2_relax
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    bl __aarch64_cas2_relax
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    bl __aarch64_cas2_acq
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    bl __aarch64_cas2_acq
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    bl __aarch64_cas2_acq
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    bl __aarch64_cas2_acq
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    bl __aarch64_cas2_acq
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    bl __aarch64_cas2_acq
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    bl __aarch64_cas2_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    bl __aarch64_cas2_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    bl __aarch64_cas4_relax
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    bl __aarch64_cas4_relax
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    bl __aarch64_cas4_acq
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    bl __aarch64_cas4_acq
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    bl __aarch64_cas4_acq
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    bl __aarch64_cas4_acq
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    bl __aarch64_cas4_acq
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    bl __aarch64_cas4_acq
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    bl __aarch64_cas4_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    bl __aarch64_cas4_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    bl __aarch64_cas8_relax
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    bl __aarch64_cas8_relax
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    bl __aarch64_cas8_acq
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    bl __aarch64_cas8_acq
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    bl __aarch64_cas8_acq
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    bl __aarch64_cas8_acq
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    bl __aarch64_cas8_acq
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    bl __aarch64_cas8_acq
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    bl __aarch64_cas8_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    bl __aarch64_cas8_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    bl __aarch64_cas16_relax
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    bl __aarch64_cas16_relax
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    bl __aarch64_cas16_acq
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    bl __aarch64_cas16_acq
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    bl __aarch64_cas16_acq
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    bl __aarch64_cas16_acq
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    bl __aarch64_cas16_acq
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    bl __aarch64_cas16_acq
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    bl __aarch64_cas16_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    bl __aarch64_cas16_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    bl __aarch64_cas1_relax
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    bl __aarch64_cas1_relax
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    bl __aarch64_cas1_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    bl __aarch64_cas1_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc.ll
new file mode 100644
index 000000000000..a432163264d9
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc3.ll
new file mode 100644
index 000000000000..595bbaba4457
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-rcpc3.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8_1a.ll
new file mode 100644
index 000000000000..8d303f769710
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8_1a.ll
@@ -0,0 +1,2494 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8a.ll
new file mode 100644
index 000000000000..891e374812a1
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-cmpxchg-v8a.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stxp w8, x2, x3, [x4]
+; -O0:    stxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x0, x1, [x4]
+; -O0:    cmp x0, x9
+; -O0:    cmp x1, x10
+; -O0:    stlxp w8, x2, x3, [x4]
+; -O0:    stlxp w8, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x0
+; -O1:    cmp x9, x1
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64-fence.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64-fence.ll
new file mode 100644
index 000000000000..db00aca9110b
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64-fence.ll
@@ -0,0 +1,47 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter "^\s*(dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64 -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @fence_acquire() {
+; CHECK-LABEL: fence_acquire:
+; CHECK:    dmb ishld
+    fence acquire
+    ret void
+}
+
+define dso_local void @fence_release() {
+; CHECK-LABEL: fence_release:
+; CHECK:    dmb ish
+    fence release
+    ret void
+}
+
+define dso_local void @fence_acq_rel() {
+; CHECK-LABEL: fence_acq_rel:
+; CHECK:    dmb ish
+    fence acq_rel
+    ret void
+}
+
+define dso_local void @fence_seq_cst() {
+; CHECK-LABEL: fence_seq_cst:
+; CHECK:    dmb ish
+    fence seq_cst
+    ret void
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse128.ll
new file mode 100644
index 000000000000..a54ad9dfcd9f
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse128.ll
@@ -0,0 +1,567 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered_const:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic_const:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire:
+; CHECK:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire_const:
+; CHECK:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst:
+; CHECK:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; CHECK:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse2.ll
new file mode 100644
index 000000000000..2fd70537a394
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-lse2.ll
@@ -0,0 +1,571 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered:
+; CHECK:    ldp x1, x0, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered_const:
+; CHECK:    ldp x1, x0, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic:
+; CHECK:    ldp x1, x0, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic_const:
+; CHECK:    ldp x1, x0, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire:
+; CHECK:    ldp x1, x0, [x0]
+; CHECK:    dmb ishld
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire_const:
+; CHECK:    ldp x1, x0, [x0]
+; CHECK:    dmb ishld
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst:
+; CHECK:    ldp x1, x0, [x0]
+; CHECK:    dmb ish
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; CHECK:    ldp x1, x0, [x0]
+; CHECK:    dmb ish
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-outline_atomics.ll
new file mode 100644
index 000000000000..7a2dfd84dcad
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-outline_atomics.ll
@@ -0,0 +1,596 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    bl __aarch64_cas16_relax
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    bl __aarch64_cas16_relax
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    bl __aarch64_cas16_relax
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    bl __aarch64_cas16_relax
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    bl __aarch64_cas16_acq
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    bl __aarch64_cas16_acq
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    bl __aarch64_cas16_acq_rel
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    bl __aarch64_cas16_acq_rel
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc.ll
new file mode 100644
index 000000000000..6703827be146
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc.ll
@@ -0,0 +1,628 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc3.ll
new file mode 100644
index 000000000000..517baa6f2f8b
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-rcpc3.ll
@@ -0,0 +1,628 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldaprh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldapr w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldapr x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldaprb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8_1a.ll
new file mode 100644
index 000000000000..322dcb24d6bf
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8_1a.ll
@@ -0,0 +1,567 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_unordered_const:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_monotonic_const:
+; CHECK:    casp x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire:
+; CHECK:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_acquire_const:
+; CHECK:    caspa x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst:
+; CHECK:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; CHECK:    caspal x2, x3, x2, x3, [x0]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8a.ll
new file mode 100644
index 000000000000..06caa68f3556
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-load-v8a.ll
@@ -0,0 +1,628 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @load_atomic_i8_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_aligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_unordered_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr unordered, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_monotonic_const:
+; CHECK:    ldrh w0, [x0]
+    %r = load atomic i16, ptr %ptr monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_acquire_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_aligned_seq_cst_const:
+; CHECK:    ldarh w0, [x0]
+    %r = load atomic i16, ptr %ptr seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_unordered_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr unordered, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_monotonic_const:
+; CHECK:    ldr w0, [x0]
+    %r = load atomic i32, ptr %ptr monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_acquire_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_aligned_seq_cst_const:
+; CHECK:    ldar w0, [x0]
+    %r = load atomic i32, ptr %ptr seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_unordered_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr unordered, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_monotonic_const:
+; CHECK:    ldr x0, [x0]
+    %r = load atomic i64, ptr %ptr monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_acquire_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_aligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_aligned_seq_cst_const:
+; CHECK:    ldar x0, [x0]
+    %r = load atomic i64, ptr %ptr seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_unordered_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_unordered_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr unordered, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_monotonic_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O0:    ldxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_monotonic_const:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_acquire_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x10, x10, [x9]
+; -O0:    stxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_acquire_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst(ptr %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_aligned_seq_cst_const(ptr readonly %ptr) {
+; -O0-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O0:    ldaxp x1, x0, [x9]
+; -O0:    cmp x1, x10
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x10, x10, [x9]
+; -O0:    stlxp w8, x1, x0, [x9]
+;
+; -O1-LABEL: load_atomic_i128_aligned_seq_cst_const:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    stlxp w9, x1, x0, [x8]
+    %r = load atomic i128, ptr %ptr seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_unordered_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr unordered, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_monotonic_const:
+; CHECK:    ldrb w0, [x0]
+    %r = load atomic i8, ptr %ptr monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_acquire_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @load_atomic_i8_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i8_unaligned_seq_cst_const:
+; CHECK:    ldarb w0, [x0]
+    %r = load atomic i8, ptr %ptr seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr unordered, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @load_atomic_i16_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i16_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i16, ptr %ptr seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr unordered, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @load_atomic_i32_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i32_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i32, ptr %ptr seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr unordered, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @load_atomic_i64_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i64_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i64, ptr %ptr seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_unordered_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_unordered_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr unordered, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_monotonic_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_monotonic_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_acquire_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_acquire_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst(ptr %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @load_atomic_i128_unaligned_seq_cst_const(ptr readonly %ptr) {
+; CHECK-LABEL: load_atomic_i128_unaligned_seq_cst_const:
+; CHECK:    bl __atomic_load
+    %r = load atomic i128, ptr %ptr seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse128.ll
new file mode 100644
index 000000000000..913abeb20f9b
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse128.ll
@@ -0,0 +1,316 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspl x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspal x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse2.ll
new file mode 100644
index 000000000000..5652cc52f024
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-lse2.ll
@@ -0,0 +1,290 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_unordered:
+; CHECK:    stp x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_monotonic:
+; CHECK:    stp x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_release:
+; CHECK:    dmb ish
+; CHECK:    stp x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_aligned_seq_cst:
+; CHECK:    dmb ish
+; CHECK:    stp x1, x0, [x2]
+; CHECK:    dmb ish
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-outline_atomics.ll
new file mode 100644
index 000000000000..c1c5c53aa7df
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-outline_atomics.ll
@@ -0,0 +1,308 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc.ll
new file mode 100644
index 000000000000..d1047d84e295
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc.ll
@@ -0,0 +1,324 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc3.ll
new file mode 100644
index 000000000000..8e067fc74b56
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-rcpc3.ll
@@ -0,0 +1,324 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8_1a.ll
new file mode 100644
index 000000000000..6a51222e3109
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8_1a.ll
@@ -0,0 +1,316 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    casp x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspl x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x2]
+; -O1:    caspal x6, x7, x0, x1, [x2]
+; -O1:    cmp x6, x4
+; -O1:    ccmp x7, x5, #0, eq
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8a.ll
new file mode 100644
index 000000000000..1a79c7335514
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomic-store-v8a.ll
@@ -0,0 +1,324 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @store_atomic_i8_aligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_aligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_aligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_unordered:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr unordered, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_monotonic:
+; CHECK:    strh w0, [x1]
+    store atomic i16 %value, ptr %ptr monotonic, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_release:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr release, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i16_aligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_aligned_seq_cst:
+; CHECK:    stlrh w0, [x1]
+    store atomic i16 %value, ptr %ptr seq_cst, align 2
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_unordered:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr unordered, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_monotonic:
+; CHECK:    str w0, [x1]
+    store atomic i32 %value, ptr %ptr monotonic, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_release:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr release, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i32_aligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_aligned_seq_cst:
+; CHECK:    stlr w0, [x1]
+    store atomic i32 %value, ptr %ptr seq_cst, align 4
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_unordered:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr unordered, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_monotonic:
+; CHECK:    str x0, [x1]
+    store atomic i64 %value, ptr %ptr monotonic, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_release:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr release, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i64_aligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_aligned_seq_cst:
+; CHECK:    stlr x0, [x1]
+    store atomic i64 %value, ptr %ptr seq_cst, align 8
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_unordered(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_unordered:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_unordered:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr unordered, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_monotonic(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_monotonic:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_monotonic:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr monotonic, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_release(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_release:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_release:
+; -O1:    ldxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr release, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i128_aligned_seq_cst(i128 %value, ptr %ptr) {
+; -O0-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: store_atomic_i128_aligned_seq_cst:
+; -O1:    ldaxp xzr, x8, [x2]
+; -O1:    stlxp w8, x1, x0, [x2]
+    store atomic i128 %value, ptr %ptr seq_cst, align 16
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_unordered(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_unordered:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_monotonic(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_monotonic:
+; CHECK:    strb w0, [x1]
+    store atomic i8 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_release(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_release:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i8_unaligned_seq_cst(i8 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i8_unaligned_seq_cst:
+; CHECK:    stlrb w0, [x1]
+    store atomic i8 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_unordered(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_monotonic(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_release(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i16_unaligned_seq_cst(i16 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i16 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_unordered(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_monotonic(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_release(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i32_unaligned_seq_cst(i32 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i32 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_unordered(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_monotonic(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_release(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i64_unaligned_seq_cst(i64 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i64 %value, ptr %ptr seq_cst, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_unordered(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_unordered:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr unordered, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_monotonic(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr monotonic, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_release(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_release:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr release, align 1
+    ret void
+}
+
+define dso_local void @store_atomic_i128_unaligned_seq_cst(i128 %value, ptr %ptr) {
+; CHECK-LABEL: store_atomic_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_store
+    store atomic i128 %value, ptr %ptr seq_cst, align 1
+    ret void
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse128.ll
new file mode 100644
index 000000000000..9d4222ec6ade
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse128.ll
@@ -0,0 +1,6289 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; CHECK:    swph w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; CHECK:    swpah w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_release:
+; CHECK:    swplh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; CHECK:    swp w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; CHECK:    swpa w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_release:
+; CHECK:    swpl w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; CHECK:    swp x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; CHECK:    swpa x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_release:
+; CHECK:    swpl x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    casp x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspa x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspl x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; CHECK:    ldaddh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acquire:
+; CHECK:    ldaddah w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_release:
+; CHECK:    ldaddlh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; CHECK:    ldadd w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acquire:
+; CHECK:    ldadda w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_release:
+; CHECK:    ldaddl w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; CHECK:    ldadd x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acquire:
+; CHECK:    ldadda x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_release:
+; CHECK:    ldaddl x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddab w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddlb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldaddh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddah w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaddah w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddlh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldaddlh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldadd w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldadd w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldadda w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldadda w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddl w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldaddl w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddal w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddal w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x8, x8, x1
+; -O0:    ldadd x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldadd x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x8, x8, x1
+; -O0:    ldadda x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldadda x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x8, x8, x1
+; -O0:    ldaddl x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldaddl x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x8, x8, x1
+; -O0:    ldaddal x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x8, x8, x1
+; -O0:    ldaddal x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddab w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddlb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrah w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclr w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclra w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrl w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclr x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acquire:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclra x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_release:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclrl x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casab w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    caslb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    cash w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cash w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casah w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casah w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    caslh w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalh w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalh w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    cas w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cas w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casa w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casa w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casl w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casl w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casal w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casal w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    cas x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    cas x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casa x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casa x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casl x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casl x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casal x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casal x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    casp x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspa x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspl x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casab w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    caslb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; CHECK:    ldseth w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acquire:
+; CHECK:    ldsetah w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_release:
+; CHECK:    ldsetlh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; CHECK:    ldset w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acquire:
+; CHECK:    ldseta w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_release:
+; CHECK:    ldsetl w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; CHECK:    ldset x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acquire:
+; CHECK:    ldseta x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_release:
+; CHECK:    ldsetl x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; CHECK:    ldeorh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; CHECK:    ldeorah w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_release:
+; CHECK:    ldeorlh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; CHECK:    ldeor w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; CHECK:    ldeora w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_release:
+; CHECK:    ldeorl w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; CHECK:    ldeor x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; CHECK:    ldeora x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_release:
+; CHECK:    ldeorl x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; CHECK:    ldsmaxh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acquire:
+; CHECK:    ldsmaxah w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_release:
+; CHECK:    ldsmaxlh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; CHECK:    ldsmax w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acquire:
+; CHECK:    ldsmaxa w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_release:
+; CHECK:    ldsmaxl w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; CHECK:    ldsmax x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acquire:
+; CHECK:    ldsmaxa x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_release:
+; CHECK:    ldsmaxl x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; CHECK:    ldsminh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acquire:
+; CHECK:    ldsminah w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_release:
+; CHECK:    ldsminlh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; CHECK:    ldsmin w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acquire:
+; CHECK:    ldsmina w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_release:
+; CHECK:    ldsminl w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; CHECK:    ldsmin x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acquire:
+; CHECK:    ldsmina x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_release:
+; CHECK:    ldsminl x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; CHECK:    ldumaxh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; CHECK:    ldumaxah w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_release:
+; CHECK:    ldumaxlh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; CHECK:    ldumax w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; CHECK:    ldumaxa w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_release:
+; CHECK:    ldumaxl w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; CHECK:    ldumax x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; CHECK:    ldumaxa x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_release:
+; CHECK:    ldumaxl x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; CHECK:    lduminh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; CHECK:    lduminah w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_release:
+; CHECK:    lduminlh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; CHECK:    ldumin w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; CHECK:    ldumina w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_release:
+; CHECK:    lduminl w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; CHECK:    ldumin x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; CHECK:    ldumina x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_release:
+; CHECK:    lduminl x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse2.ll
new file mode 100644
index 000000000000..926827e6a875
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse2.ll
@@ -0,0 +1,9094 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-outline_atomics.ll
new file mode 100644
index 000000000000..d8ffc1ad00f8
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-outline_atomics.ll
@@ -0,0 +1,7399 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_swp1_relax
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; CHECK:    bl __aarch64_swp1_acq
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_release:
+; CHECK:    bl __aarch64_swp1_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_swp2_relax
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; CHECK:    bl __aarch64_swp2_acq
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_release:
+; CHECK:    bl __aarch64_swp2_rel
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp2_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp2_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_swp4_relax
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; CHECK:    bl __aarch64_swp4_acq
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_release:
+; CHECK:    bl __aarch64_swp4_rel
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp4_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp4_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_swp8_relax
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; CHECK:    bl __aarch64_swp8_acq
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_release:
+; CHECK:    bl __aarch64_swp8_rel
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_swp8_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_swp8_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_swp1_relax
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_swp1_acq
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; CHECK:    bl __aarch64_swp1_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_swp1_acq_rel
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_release:
+; CHECK:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd2_relax
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd2_acq
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_release:
+; CHECK:    bl __aarch64_ldadd2_rel
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd4_relax
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd4_acq
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_release:
+; CHECK:    bl __aarch64_ldadd4_rel
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_ldadd8_relax
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acquire:
+; CHECK:    bl __aarch64_ldadd8_acq
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_release:
+; CHECK:    bl __aarch64_ldadd8_rel
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x3, x1, x9
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x3, x1, x9
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x3, x1, x9
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x3, x1, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x3, x1, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_release:
+; CHECK:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_relax
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_relax
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd2_relax
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_acq
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    bl __aarch64_ldadd2_acq
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_rel
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    bl __aarch64_ldadd2_rel
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd2_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd2_acq_rel
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_relax
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd4_relax
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_acq
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    bl __aarch64_ldadd4_acq
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_rel
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    bl __aarch64_ldadd4_rel
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd4_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd4_acq_rel
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_relax
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    bl __aarch64_ldadd8_relax
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_acq
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    bl __aarch64_ldadd8_acq
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_rel
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    bl __aarch64_ldadd8_rel
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x0, x8, x9
+; -O0:    bl __aarch64_ldadd8_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    bl __aarch64_ldadd8_acq_rel
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x3, x1, x9
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x3, x1, x9
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x3, x1, x9
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x3, x1, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x3, x1, x9
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_relax
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    bl __aarch64_ldadd1_relax
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    bl __aarch64_ldadd1_acq
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    bl __aarch64_ldadd1_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w0, w8, w9
+; -O0:    bl __aarch64_ldadd1_acq_rel
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    bl __aarch64_ldadd1_acq_rel
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_relax
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_relax
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_rel
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_relax
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_relax
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_acq
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_acq
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_rel
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_rel
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_acq_rel
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr2_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr2_acq_rel
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_relax
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_relax
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_acq
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_acq
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_rel
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_rel
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_acq_rel
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr4_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr4_acq_rel
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_relax
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_relax
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_acq
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_acq
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_rel
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_rel
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_acq_rel
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    mvn x0, x8
+; -O0:    bl __aarch64_ldclr8_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    mvn x0, x1
+; -O1:    bl __aarch64_ldclr8_acq_rel
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x2, x0, x9
+; -O0:    and x3, x1, x8
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x2, x0, x9
+; -O0:    and x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x2, x0, x9
+; -O0:    and x3, x1, x8
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x2, x0, x9
+; -O0:    and x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x2, x0, x9
+; -O0:    and x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_relax
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_relax
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_rel
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    mvn w0, w8
+; -O0:    bl __aarch64_ldclr1_acq_rel
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    mvn w0, w1
+; -O1:    bl __aarch64_ldclr1_acq_rel
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas2_relax
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas2_acq
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas2_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas4_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas4_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas4_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x8, x0, x8
+; -O0:    mvn x1, x8
+; -O0:    bl __aarch64_cas8_relax
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x8, x0, x8
+; -O0:    mvn x1, x8
+; -O0:    bl __aarch64_cas8_acq
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x8, x0, x8
+; -O0:    mvn x1, x8
+; -O0:    bl __aarch64_cas8_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x8, x0, x8
+; -O0:    mvn x1, x8
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x8, x0, x8
+; -O0:    mvn x1, x8
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x8, x1, x8
+; -O0:    and x9, x0, x9
+; -O0:    mvn x2, x9
+; -O0:    mvn x3, x8
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x8, x1, x8
+; -O0:    and x9, x0, x9
+; -O0:    mvn x2, x9
+; -O0:    mvn x3, x8
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x8, x1, x8
+; -O0:    and x9, x0, x9
+; -O0:    mvn x2, x9
+; -O0:    mvn x3, x8
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x8, x1, x8
+; -O0:    and x9, x0, x9
+; -O0:    mvn x2, x9
+; -O0:    mvn x3, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x8, x1, x8
+; -O0:    and x9, x0, x9
+; -O0:    mvn x2, x9
+; -O0:    mvn x3, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w0, w8
+; -O0:    mvn w1, w8
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset1_relax
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acquire:
+; CHECK:    bl __aarch64_ldset1_acq
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_release:
+; CHECK:    bl __aarch64_ldset1_rel
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset2_relax
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acquire:
+; CHECK:    bl __aarch64_ldset2_acq
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_release:
+; CHECK:    bl __aarch64_ldset2_rel
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset2_acq_rel
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset2_acq_rel
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset4_relax
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acquire:
+; CHECK:    bl __aarch64_ldset4_acq
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_release:
+; CHECK:    bl __aarch64_ldset4_rel
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset4_acq_rel
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset4_acq_rel
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_ldset8_relax
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acquire:
+; CHECK:    bl __aarch64_ldset8_acq
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_release:
+; CHECK:    bl __aarch64_ldset8_rel
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldset8_acq_rel
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldset8_acq_rel
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x2, x0, x9
+; -O0:    orr x3, x1, x8
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x2, x0, x9
+; -O0:    orr x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x2, x0, x9
+; -O0:    orr x3, x1, x8
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x2, x0, x9
+; -O0:    orr x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x2, x0, x9
+; -O0:    orr x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_ldset1_relax
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_ldset1_acq
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_release:
+; CHECK:    bl __aarch64_ldset1_rel
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_ldset1_acq_rel
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor1_relax
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor1_acq
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_release:
+; CHECK:    bl __aarch64_ldeor1_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor2_relax
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor2_acq
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_release:
+; CHECK:    bl __aarch64_ldeor2_rel
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor2_acq_rel
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor2_acq_rel
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor4_relax
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor4_acq
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_release:
+; CHECK:    bl __aarch64_ldeor4_rel
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor4_acq_rel
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor4_acq_rel
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; CHECK:    bl __aarch64_ldeor8_relax
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; CHECK:    bl __aarch64_ldeor8_acq
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_release:
+; CHECK:    bl __aarch64_ldeor8_rel
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor8_acq_rel
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor8_acq_rel
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x2, x0, x9
+; -O0:    eor x3, x1, x8
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x2, x0, x9
+; -O0:    eor x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x2, x0, x9
+; -O0:    eor x3, x1, x8
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x2, x0, x9
+; -O0:    eor x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x2, x0, x9
+; -O0:    eor x3, x1, x8
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; CHECK:    bl __aarch64_ldeor1_relax
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; CHECK:    bl __aarch64_ldeor1_acq
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_release:
+; CHECK:    bl __aarch64_ldeor1_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; CHECK:    bl __aarch64_ldeor1_acq_rel
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas2_relax
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas2_acq
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas2_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas4_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas4_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas4_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, gt
+; -O0:    bl __aarch64_cas8_relax
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, gt
+; -O0:    bl __aarch64_cas8_acq
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, gt
+; -O0:    bl __aarch64_cas8_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, gt
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, gt
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lt
+; -O0:    csel x3, x1, x8, lt
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lt
+; -O0:    csel x3, x1, x8, lt
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lt
+; -O0:    csel x3, x1, x8, lt
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lt
+; -O0:    csel x3, x1, x8, lt
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lt
+; -O0:    csel x3, x1, x8, lt
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, gt
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas2_relax
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas2_acq
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas2_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w9, w0
+; -O0:    subs w9, w9, w8, sxth
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8, uxth
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas4_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas4_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas4_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, le
+; -O0:    bl __aarch64_cas8_relax
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, le
+; -O0:    bl __aarch64_cas8_acq
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, le
+; -O0:    bl __aarch64_cas8_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, le
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, le
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, ge
+; -O0:    csel x3, x1, x8, ge
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, ge
+; -O0:    csel x3, x1, x8, ge
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, ge
+; -O0:    csel x3, x1, x8, ge
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, ge
+; -O0:    csel x3, x1, x8, ge
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, ge
+; -O0:    csel x3, x1, x8, ge
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w9, w0
+; -O0:    subs w9, w9, w8, sxtb
+; -O0:    csel w1, w0, w8, le
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8, uxtb
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas2_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas2_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas2_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas4_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas4_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas4_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, hi
+; -O0:    bl __aarch64_cas8_relax
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, hi
+; -O0:    bl __aarch64_cas8_acq
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, hi
+; -O0:    bl __aarch64_cas8_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, hi
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, hi
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lo
+; -O0:    csel x3, x1, x8, lo
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lo
+; -O0:    csel x3, x1, x8, lo
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lo
+; -O0:    csel x3, x1, x8, lo
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lo
+; -O0:    csel x3, x1, x8, lo
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, lo
+; -O0:    csel x3, x1, x8, lo
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, hi
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas2_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas2_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas2_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    and w9, w0, #0xffff
+; -O0:    subs w9, w9, w8, uxth
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas2_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas4_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas4_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas4_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w9, w0, w8
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas4_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, ls
+; -O0:    bl __aarch64_cas8_relax
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, ls
+; -O0:    bl __aarch64_cas8_acq
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, ls
+; -O0:    bl __aarch64_cas8_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, ls
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x9, x0, x8
+; -O0:    csel x1, x0, x8, ls
+; -O0:    bl __aarch64_cas8_acq_rel
+; -O0:    subs x8, x0, x8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, hs
+; -O0:    csel x3, x1, x8, hs
+; -O0:    bl __aarch64_cas16_relax
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, hs
+; -O0:    csel x3, x1, x8, hs
+; -O0:    bl __aarch64_cas16_acq
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, hs
+; -O0:    csel x3, x1, x8, hs
+; -O0:    bl __aarch64_cas16_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, hs
+; -O0:    csel x3, x1, x8, hs
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x10, x8, x1
+; -O0:    csel x2, x0, x9, hs
+; -O0:    csel x3, x1, x8, hs
+; -O0:    bl __aarch64_cas16_acq_rel
+; -O0:    subs x10, x10, x11
+; -O0:    ccmp x8, x9, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_relax
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_acq
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w0, #0xff
+; -O0:    subs w9, w9, w8, uxtb
+; -O0:    csel w1, w0, w8, ls
+; -O0:    bl __aarch64_cas1_acq_rel
+; -O0:    subs w8, w0, w8
+; -O0:    subs w8, w8, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc.ll
new file mode 100644
index 000000000000..e1fa58928a3d
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc.ll
@@ -0,0 +1,9094 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc3.ll
new file mode 100644
index 000000000000..ec0300264b06
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc3.ll
@@ -0,0 +1,9094 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8_1a.ll
new file mode 100644
index 000000000000..b126100b7495
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8_1a.ll
@@ -0,0 +1,6289 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; CHECK:    swph w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; CHECK:    swpah w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_release:
+; CHECK:    swplh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; CHECK:    swpalh w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; CHECK:    swp w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; CHECK:    swpa w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_release:
+; CHECK:    swpl w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; CHECK:    swpal w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; CHECK:    swp x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; CHECK:    swpa x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_release:
+; CHECK:    swpl x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; CHECK:    swpal x1, x0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    casp x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspa x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspl x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    caspal x4, x5, x2, x3, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; CHECK:    swpb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; CHECK:    swpab w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; CHECK:    swplb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; CHECK:    swpalb w1, w0, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; CHECK:    ldaddh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acquire:
+; CHECK:    ldaddah w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_release:
+; CHECK:    ldaddlh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; CHECK:    ldaddalh w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; CHECK:    ldadd w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acquire:
+; CHECK:    ldadda w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_release:
+; CHECK:    ldaddl w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; CHECK:    ldaddal w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; CHECK:    ldadd x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acquire:
+; CHECK:    ldadda x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_release:
+; CHECK:    ldaddl x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; CHECK:    ldaddal x1, x0, [x0]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    adds x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; CHECK:    ldaddb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; CHECK:    ldaddab w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_release:
+; CHECK:    ldaddlb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; CHECK:    ldaddalb w1, w0, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddab w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddlb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldaddh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddah w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaddah w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddlh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldaddlh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalh w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaddalh w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldadd w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldadd w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldadda w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldadda w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddl w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldaddl w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddal w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddal w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaddal w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x8, x8, x1
+; -O0:    ldadd x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldadd x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x8, x8, x1
+; -O0:    ldadda x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldadda x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x8, x8, x1
+; -O0:    ldaddl x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldaddl x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x8, x8, x1
+; -O0:    ldaddal x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x8, x8, x1
+; -O0:    ldaddal x8, x0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaddal x8, x0, [x0]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    subs x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldaddb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddab w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaddab w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddlb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldaddlb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w8, w8, w1
+; -O0:    ldaddalb w8, w0, [x0]
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaddalb w8, w0, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrah w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralh w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclr w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclra w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrl w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclral w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclr x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acquire:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclra x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_release:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclrl x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; CHECK:    mvn x8, x1
+; CHECK:    ldclral x8, x0, [x0]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x2, x11, x12
+; -O0:    and x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrab w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_release:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclrlb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; CHECK:    mvn w8, w1
+; CHECK:    ldclralb w8, w0, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casab w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    caslb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    cash w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cash w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casah w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casah w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    caslh w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalh w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalh w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalh w9, w10, [x0]
+; -O1:    cmp w9, w8, uxth
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    cas w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    cas w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casa w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casa w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casl w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casl w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casal w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casal w8, w10, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casal w9, w10, [x0]
+; -O1:    cmp w9, w8
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    cas x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    cas x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casa x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casa x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casl x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casl x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casal x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x10, x8
+; -O0:    casal x8, x10, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    and x10, x8, x1
+; -O1:    mvn x10, x10
+; -O1:    casal x9, x10, [x0]
+; -O1:    cmp x9, x8
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    casp x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspa x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspl x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x9, x10, x9
+; -O0:    and x12, x11, x12
+; -O0:    mvn x2, x12
+; -O0:    mvn x9, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    and x8, x4, x2
+; -O1:    and x9, x7, x3
+; -O1:    mvn x10, x8
+; -O1:    mvn x11, x9
+; -O1:    caspal x4, x5, x10, x11, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casab w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casab w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    caslb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    caslb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w10, w8
+; -O0:    casalb w8, w10, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    and w10, w8, w1
+; -O1:    mvn w10, w10
+; -O1:    casalb w9, w10, [x0]
+; -O1:    cmp w9, w8, uxtb
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; CHECK:    ldseth w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acquire:
+; CHECK:    ldsetah w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_release:
+; CHECK:    ldsetlh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; CHECK:    ldsetalh w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; CHECK:    ldset w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acquire:
+; CHECK:    ldseta w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_release:
+; CHECK:    ldsetl w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; CHECK:    ldsetal w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; CHECK:    ldset x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acquire:
+; CHECK:    ldseta x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_release:
+; CHECK:    ldsetl x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; CHECK:    ldsetal x1, x0, [x0]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x2, x11, x12
+; -O0:    orr x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    orr x8, x4, x2
+; -O1:    orr x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; CHECK:    ldsetb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; CHECK:    ldsetab w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_release:
+; CHECK:    ldsetlb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; CHECK:    ldsetalb w1, w0, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; CHECK:    ldeorh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; CHECK:    ldeorah w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_release:
+; CHECK:    ldeorlh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; CHECK:    ldeoralh w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; CHECK:    ldeor w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; CHECK:    ldeora w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_release:
+; CHECK:    ldeorl w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; CHECK:    ldeoral w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; CHECK:    ldeor x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; CHECK:    ldeora x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_release:
+; CHECK:    ldeorl x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; CHECK:    ldeoral x1, x0, [x0]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x2, x11, x12
+; -O0:    eor x9, x10, x9
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    eor x8, x4, x2
+; -O1:    eor x9, x7, x3
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; CHECK:    ldeorb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; CHECK:    ldeorab w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_release:
+; CHECK:    ldeorlb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; CHECK:    ldeoralb w1, w0, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; CHECK:    ldsmaxh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acquire:
+; CHECK:    ldsmaxah w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_release:
+; CHECK:    ldsmaxlh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; CHECK:    ldsmaxalh w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; CHECK:    ldsmax w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acquire:
+; CHECK:    ldsmaxa w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_release:
+; CHECK:    ldsmaxl w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; CHECK:    ldsmaxal w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; CHECK:    ldsmax x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acquire:
+; CHECK:    ldsmaxa x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_release:
+; CHECK:    ldsmaxl x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; CHECK:    ldsmaxal x1, x0, [x0]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lt
+; -O0:    csel x2, x11, x12, lt
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lt
+; -O1:    csel x8, x4, x2, lt
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; CHECK:    ldsmaxb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; CHECK:    ldsmaxab w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_release:
+; CHECK:    ldsmaxlb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; CHECK:    ldsmaxalb w1, w0, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; CHECK:    ldsminh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acquire:
+; CHECK:    ldsminah w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_release:
+; CHECK:    ldsminlh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; CHECK:    ldsminalh w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; CHECK:    ldsmin w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acquire:
+; CHECK:    ldsmina w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_release:
+; CHECK:    ldsminl w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; CHECK:    ldsminal w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; CHECK:    ldsmin x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acquire:
+; CHECK:    ldsmina x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_release:
+; CHECK:    ldsminl x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; CHECK:    ldsminal x1, x0, [x0]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, ge
+; -O0:    csel x2, x11, x12, ge
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, ge
+; -O1:    csel x8, x4, x2, ge
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; CHECK:    ldsminb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; CHECK:    ldsminab w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_release:
+; CHECK:    ldsminlb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; CHECK:    ldsminalb w1, w0, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; CHECK:    ldumaxh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; CHECK:    ldumaxah w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_release:
+; CHECK:    ldumaxlh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; CHECK:    ldumaxalh w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; CHECK:    ldumax w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; CHECK:    ldumaxa w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_release:
+; CHECK:    ldumaxl w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; CHECK:    ldumaxal w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; CHECK:    ldumax x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; CHECK:    ldumaxa x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_release:
+; CHECK:    ldumaxl x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; CHECK:    ldumaxal x1, x0, [x0]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, lo
+; -O0:    csel x2, x11, x12, lo
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, lo
+; -O1:    csel x8, x4, x2, lo
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; CHECK:    ldumaxb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; CHECK:    ldumaxab w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_release:
+; CHECK:    ldumaxlb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; CHECK:    ldumaxalb w1, w0, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; CHECK:    lduminh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; CHECK:    lduminah w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_release:
+; CHECK:    lduminlh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; CHECK:    lduminalh w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; CHECK:    ldumin w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; CHECK:    ldumina w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_release:
+; CHECK:    lduminl w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; CHECK:    lduminal w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; CHECK:    ldumin x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; CHECK:    ldumina x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_release:
+; CHECK:    lduminl x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; CHECK:    lduminal x1, x0, [x0]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    casp x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    casp x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspa x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspa x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspl x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspl x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x13, x9, x10
+; -O0:    csel x9, x10, x9, hs
+; -O0:    csel x2, x11, x12, hs
+; -O0:    caspal x0, x1, x2, x3, [x8]
+; -O0:    subs x11, x9, x11
+; -O0:    ccmp x8, x10, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldp x4, x5, [x0]
+; -O1:    cmp x3, x7
+; -O1:    csel x9, x7, x3, hs
+; -O1:    csel x8, x4, x2, hs
+; -O1:    caspal x4, x5, x8, x9, [x0]
+; -O1:    cmp x4, x6
+; -O1:    ccmp x5, x7, #0, eq
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; CHECK:    lduminb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; CHECK:    lduminab w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_release:
+; CHECK:    lduminlb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; CHECK-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; CHECK:    lduminalb w1, w0, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8a.ll
new file mode 100644
index 000000000000..d52b800dac34
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8a.ll
@@ -0,0 +1,9094 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    stlxrh w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_monotonic:
+; -O1:    ldxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acquire:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_release:
+; -O1:    ldxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_acq_rel:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i32_aligned_seq_cst:
+; -O1:    ldaxr w0, [x8]
+; -O1:    stlxr w9, w1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    stlxr w9, x1, [x8]
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_monotonic:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acquire:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_release:
+; -O1:    ldxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xchg_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x8, [x0]
+; -O1:    stlxp w9, x3, x2, [x0]
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xchg_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xchg_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    stlxrb w9, w1, [x0]
+    %r = atomicrmw xchg ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_release(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xchg_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i16_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_release(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xchg_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i32_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_release(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xchg_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i64_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_monotonic:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acquire:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_release(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_release:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_acq_rel:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xchg_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; CHECK-LABEL: atomicrmw_xchg_i128_unaligned_seq_cst:
+; CHECK:    bl __atomic_exchange
+    %r = atomicrmw xchg ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_release:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O0:    add x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    add x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_release:
+; -O0:    adds x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O0:    adds x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_add_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    adds x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_add_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O0:    add w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_add_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    add w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw add ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_add_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i16_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_monotonic:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acquire:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_release:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_acq_rel:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_add_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O0:    add w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i32_unaligned_seq_cst:
+; -O1:    add w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_monotonic:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acquire:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_release:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_acq_rel:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_add_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O0:    add x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i64_unaligned_seq_cst:
+; -O1:    add x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_add_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O0:    adds x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_add_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    adds x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw add ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O0:    subs x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    sub x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O0:    subs x14, x11, x10
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O0:    subs x14, x11, x10
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_sub_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    subs x9, x1, x3
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_sub_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O0:    subs w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_sub_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    sub w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw sub ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_sub_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i16_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_monotonic:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acquire:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_release:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_acq_rel:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_sub_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O0:    subs w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i32_unaligned_seq_cst:
+; -O1:    sub w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_monotonic:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acquire:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_release:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_acq_rel:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_sub_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O0:    subs x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i64_unaligned_seq_cst:
+; -O1:    sub x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_sub_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O0:    subs x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_sub_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    subs x8, x1, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw sub ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_release:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O0:    and x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_release:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O0:    and x15, x13, x10
+; -O0:    and x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_and_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    and x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_and_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O0:    and w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_and_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw and ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_and_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_and_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_and_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_and_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_and_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw and ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x12, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    and x9, x0, x1
+; -O1:    mvn x9, x9
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O0:    and x8, x11, x8
+; -O0:    and x10, x13, x10
+; -O0:    mvn x15, x10
+; -O0:    mvn x14, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_nand_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    and x9, x1, x3
+; -O1:    mvn x9, x9
+; -O1:    and x10, x0, x2
+; -O1:    mvn x10, x10
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_nand_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w12, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_nand_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    and w9, w8, w1
+; -O1:    mvn w9, w9
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw nand ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_nand_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_monotonic:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acquire:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_release:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_acq_rel:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_nand_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O0:    and w8, w9, w8
+; -O0:    mvn w8, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i32_unaligned_seq_cst:
+; -O1:    and w8, w0, w19
+; -O1:    mvn w8, w8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_monotonic:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acquire:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_release:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_acq_rel:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_nand_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O0:    and x8, x9, x8
+; -O0:    mvn x8, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i64_unaligned_seq_cst:
+; -O1:    and x8, x0, x19
+; -O1:    mvn x8, x8
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_nand_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O0:    and x9, x11, x9
+; -O0:    and x8, x10, x8
+; -O0:    mvn x8, x8
+; -O0:    mvn x9, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_nand_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    and x8, x1, x19
+; -O1:    and x9, x0, x20
+; -O1:    mvn x8, x8
+; -O1:    mvn x9, x9
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw nand ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_release:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O0:    orr x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    orr x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_release:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O0:    orr x15, x13, x10
+; -O0:    orr x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_or_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    orr x9, x1, x3
+; -O1:    orr x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_or_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O0:    orr w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_or_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    orr w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw or ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_or_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i16_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_monotonic:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acquire:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_release:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_acq_rel:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_or_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O0:    orr w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i32_unaligned_seq_cst:
+; -O1:    orr w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_monotonic:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acquire:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_release:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_acq_rel:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_or_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O0:    orr x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i64_unaligned_seq_cst:
+; -O1:    orr x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_or_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O0:    orr x8, x11, x8
+; -O0:    orr x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_or_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    orr x8, x1, x19
+; -O1:    orr x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw or ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_aligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_monotonic:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acquire:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_release:
+; -O1:    ldxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_acq_rel:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i16_aligned_seq_cst:
+; -O1:    ldaxrh w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O0:    eor x12, x9, x8
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    eor x9, x0, x1
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O0:    eor x15, x13, x10
+; -O0:    eor x14, x11, x8
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_xor_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    eor x9, x1, x3
+; -O1:    eor x10, x0, x2
+; -O1:    stlxp w11, x9, x10, [x8]
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_monotonic:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acquire:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_release:
+; -O1:    ldxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_xor_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O0:    eor w12, w9, w8
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_xor_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w8, [x0]
+; -O1:    eor w9, w8, w1
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw xor ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_xor_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i16_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_monotonic:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acquire:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_release:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_acq_rel:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_xor_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O0:    eor w8, w9, w8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i32_unaligned_seq_cst:
+; -O1:    eor w8, w0, w19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_monotonic:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acquire:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_release:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_acq_rel:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_xor_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O0:    eor x8, x9, x8
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i64_unaligned_seq_cst:
+; -O1:    eor x8, x0, x19
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_xor_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O0:    eor x8, x11, x8
+; -O0:    eor x9, x10, x9
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_xor_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    eor x8, x1, x19
+; -O1:    eor x9, x0, x20
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw xor ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, gt
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, gt
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, gt
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lt
+; -O0:    csel x14, x11, x8, lt
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_max_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lt
+; -O1:    csel x10, x1, x3, lt
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_max_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, gt
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_max_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, gt
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw max ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_max_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_max_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_max_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, gt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, gt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_max_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lt
+; -O0:    csel x9, x10, x9, lt
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_max_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lt
+; -O1:    csel x9, x0, x20, lt
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw max ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_aligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_monotonic:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acquire:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_release:
+; -O1:    ldxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_acq_rel:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w9, uxth
+; -O0:    stlxrh w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxth
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i16_aligned_seq_cst:
+; -O1:    ldaxrh w9, [x0]
+; -O1:    sxth w8, w9
+; -O1:    cmp w8, w1, sxth
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrh w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, le
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, le
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, le
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, ge
+; -O0:    csel x14, x11, x8, ge
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_min_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, ge
+; -O1:    csel x10, x1, x3, ge
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_monotonic:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acquire:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_release:
+; -O1:    ldxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_acq_rel:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_min_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O0:    sxtb w10, w9
+; -O0:    subs w10, w10, w8, sxtb
+; -O0:    csel w12, w9, w8, le
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w9, uxtb
+; -O0:    stlxrb w10, w12, [x11]
+; -O0:    subs w9, w8, w9, uxtb
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_min_i8_unaligned_seq_cst:
+; -O1:    ldaxrb w9, [x0]
+; -O1:    sxtb w8, w9
+; -O1:    cmp w8, w1, sxtb
+; -O1:    csel w9, w9, w1, le
+; -O1:    stlxrb w10, w9, [x0]
+    %r = atomicrmw min ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_monotonic:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acquire:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_release:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_acq_rel:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_min_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O0:    sxth w10, w9
+; -O0:    subs w10, w10, w8, sxth
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i16_unaligned_seq_cst:
+; -O1:    sxth w8, w0
+; -O1:    cmp w8, w19, sxth
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_min_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_min_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, le
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, le
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_min_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, ge
+; -O0:    csel x9, x10, x9, ge
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_min_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, ge
+; -O1:    csel x9, x0, x20, ge
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw min ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, hi
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, hi
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, hi
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, hi
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, lo
+; -O0:    csel x14, x11, x8, lo
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umax_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, lo
+; -O1:    csel x10, x1, x3, lo
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umax_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, hi
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umax_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, hi
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umax ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umax_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umax_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umax_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, hi
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, hi
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umax_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, lo
+; -O0:    csel x9, x10, x9, lo
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umax_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, lo
+; -O1:    csel x9, x0, x20, lo
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umax ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_aligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_aligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_monotonic:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acquire:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_release:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_acq_rel:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 2
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_aligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O0:    and w9, w12, #0xffff
+; -O0:    subs w10, w9, w8, uxth
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrh w8, [x11]
+; -O0:    cmp w8, w12, uxth
+; -O0:    stlxrh w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i16_aligned_seq_cst:
+; -O1:    and w9, w1, #0xffff
+; -O1:    ldaxrh w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrh w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 2
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_monotonic:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acquire:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_release:
+; -O1:    ldxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_acq_rel:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 4
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_aligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w12, w9, w8, ls
+; -O0:    ldaxr w8, [x11]
+; -O0:    cmp w8, w9
+; -O0:    stlxr w10, w12, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i32_aligned_seq_cst:
+; -O1:    ldaxr w8, [x0]
+; -O1:    cmp w8, w1
+; -O1:    csel w9, w8, w1, ls
+; -O1:    stlxr w10, w9, [x0]
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 4
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_monotonic:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acquire:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_release:
+; -O1:    ldxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_acq_rel:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 8
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_aligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x12, x9, x8, ls
+; -O0:    ldaxr x8, [x11]
+; -O0:    cmp x8, x9
+; -O0:    stlxr w10, x12, [x11]
+; -O0:    subs x9, x8, x9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i64_aligned_seq_cst:
+; -O1:    ldaxr x0, [x8]
+; -O1:    cmp x0, x1
+; -O1:    csel x9, x0, x1, ls
+; -O1:    stlxr w10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 8
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_monotonic:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stxp w8, x14, x15, [x9]
+; -O0:    stxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acquire:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_release:
+; -O1:    ldxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_acq_rel:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 16
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_aligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O0:    subs x12, x8, x11
+; -O0:    csel x15, x13, x10, hs
+; -O0:    csel x14, x11, x8, hs
+; -O0:    ldaxp x10, x12, [x9]
+; -O0:    cmp x10, x11
+; -O0:    cmp x12, x13
+; -O0:    stlxp w8, x14, x15, [x9]
+; -O0:    stlxp w8, x10, x12, [x9]
+; -O0:    subs x12, x12, x13
+; -O0:    ccmp x10, x11, #0, eq
+;
+; -O1-LABEL: atomicrmw_umin_i128_aligned_seq_cst:
+; -O1:    ldaxp x1, x0, [x8]
+; -O1:    cmp x3, x1
+; -O1:    csel x9, x0, x2, hs
+; -O1:    csel x10, x1, x3, hs
+; -O1:    stlxp w11, x10, x9, [x8]
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 16
+    ret i128 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_monotonic(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_monotonic:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value monotonic, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acquire(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acquire:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acquire, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_release(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_release:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value release, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_acq_rel(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_acq_rel:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value acq_rel, align 1
+    ret i8 %r
+}
+
+define dso_local i8 @atomicrmw_umin_i8_unaligned_seq_cst(ptr %ptr, i8 %value) {
+; -O0-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O0:    and w9, w12, #0xff
+; -O0:    subs w10, w9, w8, uxtb
+; -O0:    csel w13, w12, w8, ls
+; -O0:    ldaxrb w8, [x11]
+; -O0:    cmp w8, w12, uxtb
+; -O0:    stlxrb w10, w13, [x11]
+; -O0:    subs w9, w8, w9
+; -O0:    subs w9, w9, #1
+;
+; -O1-LABEL: atomicrmw_umin_i8_unaligned_seq_cst:
+; -O1:    and w9, w1, #0xff
+; -O1:    ldaxrb w8, [x0]
+; -O1:    cmp w8, w9
+; -O1:    csel w10, w8, w9, ls
+; -O1:    stlxrb w11, w10, [x0]
+    %r = atomicrmw umin ptr %ptr, i8 %value seq_cst, align 1
+    ret i8 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_monotonic(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value monotonic, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acquire(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acquire, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_release(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_release:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value release, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_acq_rel(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_acq_rel:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value acq_rel, align 1
+    ret i16 %r
+}
+
+define dso_local i16 @atomicrmw_umin_i16_unaligned_seq_cst(ptr %ptr, i16 %value) {
+; -O0-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O0:    and w10, w9, #0xffff
+; -O0:    subs w10, w10, w8, uxth
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i16_unaligned_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    cmp w8, w19, uxth
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i16 %value seq_cst, align 1
+    ret i16 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_monotonic(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_monotonic:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value monotonic, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acquire(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acquire:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acquire, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_release(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_release:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value release, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_acq_rel(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_acq_rel:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value acq_rel, align 1
+    ret i32 %r
+}
+
+define dso_local i32 @atomicrmw_umin_i32_unaligned_seq_cst(ptr %ptr, i32 %value) {
+; -O0-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O0:    subs w10, w9, w8
+; -O0:    csel w8, w9, w8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i32_unaligned_seq_cst:
+; -O1:    cmp w0, w19
+; -O1:    csel w8, w0, w19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i32 %value seq_cst, align 1
+    ret i32 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_monotonic(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_monotonic:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value monotonic, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acquire(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acquire:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acquire, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_release(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_release:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value release, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_acq_rel(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_acq_rel:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value acq_rel, align 1
+    ret i64 %r
+}
+
+define dso_local i64 @atomicrmw_umin_i64_unaligned_seq_cst(ptr %ptr, i64 %value) {
+; -O0-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O0:    subs x10, x9, x8
+; -O0:    csel x8, x9, x8, ls
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i64_unaligned_seq_cst:
+; -O1:    cmp x0, x19
+; -O1:    csel x8, x0, x19, ls
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i64 %value seq_cst, align 1
+    ret i64 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_monotonic(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_monotonic:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value monotonic, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acquire(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acquire:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acquire, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_release(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_release:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value release, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_acq_rel(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_acq_rel:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value acq_rel, align 1
+    ret i128 %r
+}
+
+define dso_local i128 @atomicrmw_umin_i128_unaligned_seq_cst(ptr %ptr, i128 %value) {
+; -O0-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O0:    subs x12, x9, x10
+; -O0:    csel x8, x11, x8, hs
+; -O0:    csel x9, x10, x9, hs
+; -O0:    bl __atomic_compare_exchange
+;
+; -O1-LABEL: atomicrmw_umin_i128_unaligned_seq_cst:
+; -O1:    ldp x0, x1, [x0]
+; -O1:    cmp x19, x1
+; -O1:    csel x8, x1, x19, hs
+; -O1:    csel x9, x0, x20, hs
+; -O1:    bl __atomic_compare_exchange
+    %r = atomicrmw umin ptr %ptr, i128 %value seq_cst, align 1
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse128.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse128.ll
new file mode 100644
index 000000000000..9204887d0ccd
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse128.ll
@@ -0,0 +1,2494 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse2.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse2.ll
new file mode 100644
index 000000000000..b9045a189312
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-lse2.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-outline_atomics.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-outline_atomics.ll
new file mode 100644
index 000000000000..efbe019363c1
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-outline_atomics.ll
@@ -0,0 +1,2407 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; CHECK:    bl __aarch64_cas1_relax
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_relax
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; CHECK:    bl __aarch64_cas1_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; CHECK:    bl __aarch64_cas2_relax
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; CHECK:    bl __aarch64_cas2_relax
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; CHECK:    bl __aarch64_cas2_acq
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; CHECK:    bl __aarch64_cas2_acq
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; CHECK:    bl __aarch64_cas2_acq
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; CHECK:    bl __aarch64_cas2_acq
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; CHECK:    bl __aarch64_cas2_acq
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; CHECK:    bl __aarch64_cas2_acq
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; CHECK:    bl __aarch64_cas2_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; CHECK:    bl __aarch64_cas2_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __aarch64_cas2_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; CHECK:    bl __aarch64_cas4_relax
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; CHECK:    bl __aarch64_cas4_relax
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; CHECK:    bl __aarch64_cas4_acq
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; CHECK:    bl __aarch64_cas4_acq
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; CHECK:    bl __aarch64_cas4_acq
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; CHECK:    bl __aarch64_cas4_acq
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; CHECK:    bl __aarch64_cas4_acq
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; CHECK:    bl __aarch64_cas4_acq
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; CHECK:    bl __aarch64_cas4_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; CHECK:    bl __aarch64_cas4_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __aarch64_cas4_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; CHECK:    bl __aarch64_cas8_relax
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; CHECK:    bl __aarch64_cas8_relax
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; CHECK:    bl __aarch64_cas8_acq
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; CHECK:    bl __aarch64_cas8_acq
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; CHECK:    bl __aarch64_cas8_acq
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; CHECK:    bl __aarch64_cas8_acq
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; CHECK:    bl __aarch64_cas8_acq
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; CHECK:    bl __aarch64_cas8_acq
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; CHECK:    bl __aarch64_cas8_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; CHECK:    bl __aarch64_cas8_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __aarch64_cas8_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; CHECK:    bl __aarch64_cas16_relax
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; CHECK:    bl __aarch64_cas16_relax
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; CHECK:    bl __aarch64_cas16_acq
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; CHECK:    bl __aarch64_cas16_acq
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; CHECK:    bl __aarch64_cas16_acq
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; CHECK:    bl __aarch64_cas16_acq
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; CHECK:    bl __aarch64_cas16_acq
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; CHECK:    bl __aarch64_cas16_acq
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; CHECK:    bl __aarch64_cas16_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; CHECK:    bl __aarch64_cas16_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_release_acquire:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __aarch64_cas16_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; CHECK:    bl __aarch64_cas1_relax
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_relax
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; CHECK:    bl __aarch64_cas1_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __aarch64_cas1_acq_rel
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc.ll
new file mode 100644
index 000000000000..0ea53b7701a7
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc3.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc3.ll
new file mode 100644
index 000000000000..804c9c7cb10e
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-rcpc3.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8_1a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8_1a.ll
new file mode 100644
index 000000000000..d58c14285a59
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8_1a.ll
@@ -0,0 +1,2494 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; CHECK:    cash w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; CHECK:    casah w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; CHECK:    caslh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casalh w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; CHECK:    cas w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; CHECK:    casa w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; CHECK:    casl w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; CHECK:    cas x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; CHECK:    casa x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; CHECK:    casl x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; CHECK:    casal x0, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    casp x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    casp x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    caspa x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    caspa x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    caspl x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    caspl x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    caspal x2, x3, x0, x1, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    caspal x0, x1, x2, x3, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; CHECK:    casb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; CHECK:    casab w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; CHECK:    caslb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    casalb w0, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8a.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8a.ll
new file mode 100644
index 000000000000..4b397abd280d
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-cmpxchg-v8a.ll
@@ -0,0 +1,3829 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_aligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_monotonic_weak:
+; -O1:    ldxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_release_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xffff
+; -O1:    ldaxrh w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_aligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrh w0, [x2]
+; -O0:    cmp w0, w9, uxth
+; -O0:    stlxrh w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i16_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrh w8, [x2]
+; -O1:    cmp w8, w0, uxth
+; -O1:    stlxrh w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 2
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic:
+; -O1:    ldxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_monotonic_weak:
+; -O1:    ldxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_release_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_aligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr w0, [x2]
+; -O0:    cmp w0, w9
+; -O0:    stlxr w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i32_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr w8, [x2]
+; -O1:    cmp w8, w0
+; -O1:    stlxr w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 4
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic:
+; -O1:    ldxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_monotonic_weak:
+; -O1:    ldxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_release_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst:
+; -O1:    ldaxr x0, [x2]
+; -O1:    cmp x0, x8
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_aligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxr x0, [x2]
+; -O0:    cmp x0, x9
+; -O0:    stlxr w8, x1, [x2]
+;
+; -O1-LABEL: cmpxchg_i64_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxr x8, [x2]
+; -O1:    cmp x8, x0
+; -O1:    stlxr w9, x1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 8
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_monotonic_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stxp w8, x3, x2, [x4]
+; -O0:    stxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stxp w10, x8, x9, [x4]
+; -O1:    stxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acquire_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O0:    ldxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_monotonic_weak:
+; -O1:    ldxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_release_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_monotonic_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_acquire_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_aligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxp x1, x0, [x4]
+; -O0:    cmp x1, x9
+; -O0:    cmp x0, x10
+; -O0:    stlxp w8, x3, x2, [x4]
+; -O0:    stlxp w8, x1, x0, [x4]
+;
+; -O1-LABEL: cmpxchg_i128_aligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxp x8, x9, [x4]
+; -O1:    cmp x8, x1
+; -O1:    cmp x9, x0
+; -O1:    stlxp w10, x8, x9, [x4]
+; -O1:    stlxp w10, x3, x2, [x4]
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 16
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_monotonic_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_monotonic_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new monotonic seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acquire_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acquire_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acquire seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_monotonic_weak:
+; -O1:    ldxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_release_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_release_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new release seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_acq_rel_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_acq_rel_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_monotonic_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_monotonic_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst monotonic, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_acquire_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_acquire_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst acquire, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst:
+; -O1:    and w8, w0, #0xff
+; -O1:    ldaxrb w0, [x2]
+; -O1:    cmp w0, w8
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i8 @cmpxchg_i8_unaligned_seq_cst_seq_cst_weak(i8 %expected, i8 %new, ptr %ptr) {
+; -O0-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O0:    ldaxrb w0, [x2]
+; -O0:    cmp w0, w9, uxtb
+; -O0:    stlxrb w8, w1, [x2]
+;
+; -O1-LABEL: cmpxchg_i8_unaligned_seq_cst_seq_cst_weak:
+; -O1:    ldaxrb w8, [x2]
+; -O1:    cmp w8, w0, uxtb
+; -O1:    stlxrb w9, w1, [x2]
+    %pair = cmpxchg weak ptr %ptr, i8 %expected, i8 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i8, i1 } %pair, 0
+    ret i8 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_monotonic_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new monotonic seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acquire_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acquire seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_release_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new release seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_acq_rel_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_monotonic_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst monotonic, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_acquire_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst acquire, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i16 @cmpxchg_i16_unaligned_seq_cst_seq_cst_weak(i16 %expected, i16 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i16_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i16 %expected, i16 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i16, i1 } %pair, 0
+    ret i16 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_monotonic_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new monotonic seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acquire_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acquire seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_release_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new release seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_acq_rel_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_monotonic_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst monotonic, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_acquire_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst acquire, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i32 @cmpxchg_i32_unaligned_seq_cst_seq_cst_weak(i32 %expected, i32 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i32_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i32 %expected, i32 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i32, i1 } %pair, 0
+    ret i32 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_monotonic_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new monotonic seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acquire_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acquire seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_release_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new release seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_acq_rel_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_monotonic_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst monotonic, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_acquire_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst acquire, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i64 @cmpxchg_i64_unaligned_seq_cst_seq_cst_weak(i64 %expected, i64 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i64_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i64 %expected, i64 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i64, i1 } %pair, 0
+    ret i64 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_monotonic_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_monotonic_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new monotonic seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acquire_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acquire_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acquire seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_release_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_release_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new release seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_acq_rel_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_acq_rel_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new acq_rel seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_monotonic_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_monotonic_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst monotonic, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_acquire_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_acquire_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst acquire, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}
+
+define dso_local i128 @cmpxchg_i128_unaligned_seq_cst_seq_cst_weak(i128 %expected, i128 %new, ptr %ptr) {
+; CHECK-LABEL: cmpxchg_i128_unaligned_seq_cst_seq_cst_weak:
+; CHECK:    bl __atomic_compare_exchange
+    %pair = cmpxchg weak ptr %ptr, i128 %expected, i128 %new seq_cst seq_cst, align 1
+    %r = extractvalue { i128, i1 } %pair, 0
+    ret i128 %r
+}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-fence.ll b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-fence.ll
new file mode 100644
index 000000000000..7a9b5e22e46a
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/aarch64_be-fence.ll
@@ -0,0 +1,47 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --filter "^\s*(dmb)"
+; The base test file was generated by ./llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+v8.1a -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse2 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+outline-atomics -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+rcpc3 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O0 | FileCheck %s --check-prefixes=CHECK,-O0
+; RUN: llc %s -o - -verify-machineinstrs -mtriple=aarch64_be -mattr=+lse128 -O1 | FileCheck %s --check-prefixes=CHECK,-O1
+
+define dso_local void @fence_acquire() {
+; CHECK-LABEL: fence_acquire:
+; CHECK:    dmb ishld
+    fence acquire
+    ret void
+}
+
+define dso_local void @fence_release() {
+; CHECK-LABEL: fence_release:
+; CHECK:    dmb ish
+    fence release
+    ret void
+}
+
+define dso_local void @fence_acq_rel() {
+; CHECK-LABEL: fence_acq_rel:
+; CHECK:    dmb ish
+    fence acq_rel
+    ret void
+}
+
+define dso_local void @fence_seq_cst() {
+; CHECK-LABEL: fence_seq_cst:
+; CHECK:    dmb ish
+    fence seq_cst
+    ret void
+}
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; -O0: {{.*}}
+; -O1: {{.*}}

diff  --git a/llvm/test/CodeGen/AArch64/Atomics/generate-tests.py b/llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
new file mode 100755
index 000000000000..9104dab0cfda
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/Atomics/generate-tests.py
@@ -0,0 +1,277 @@
+#!/usr/bin/env python3
+import textwrap
+import enum
+import os
+"""
+Generate the tests in llvm/test/CodeGen/AArch64/Atomics. Run from top level llvm-project.
+"""
+
+TRIPLES = [
+    'aarch64',
+    'aarch64_be',
+]
+
+
+# Type name size
+class Type(enum.Enum):
+    # Value is the size in bytes
+    i8 = 1
+    i16 = 2
+    i32 = 4
+    i64 = 8
+    i128 = 16
+
+    def align(self, aligned: bool) -> int:
+        return self.value if aligned else 1
+
+    def __str__(self) -> str:
+        return self.name
+
+
+# Is this an aligned or unaligned access?
+class Aligned(enum.Enum):
+    aligned = True
+    unaligned = False
+
+    def __str__(self) -> str:
+        return self.name
+
+    def __bool__(self) -> bool:
+        return self.value
+
+
+class AtomicOrder(enum.Enum):
+    notatomic = 0
+    unordered = 1
+    monotonic = 2
+    acquire = 3
+    release = 4
+    acq_rel = 5
+    seq_cst = 6
+
+    def __str__(self) -> str:
+        return self.name
+
+
+ATOMICRMW_ORDERS = [
+    AtomicOrder.monotonic,
+    AtomicOrder.acquire,
+    AtomicOrder.release,
+    AtomicOrder.acq_rel,
+    AtomicOrder.seq_cst,
+]
+
+ATOMIC_LOAD_ORDERS = [
+    AtomicOrder.unordered,
+    AtomicOrder.monotonic,
+    AtomicOrder.acquire,
+    AtomicOrder.seq_cst,
+]
+
+ATOMIC_STORE_ORDERS = [
+    AtomicOrder.unordered,
+    AtomicOrder.monotonic,
+    AtomicOrder.release,
+    AtomicOrder.seq_cst,
+]
+
+ATOMIC_FENCE_ORDERS = [
+    AtomicOrder.acquire,
+    AtomicOrder.release,
+    AtomicOrder.acq_rel,
+    AtomicOrder.seq_cst,
+]
+
+CMPXCHG_SUCCESS_ORDERS = [
+    AtomicOrder.monotonic,
+    AtomicOrder.acquire,
+    AtomicOrder.release,
+    AtomicOrder.acq_rel,
+    AtomicOrder.seq_cst,
+]
+
+CMPXCHG_FAILURE_ORDERS = [
+    AtomicOrder.monotonic,
+    AtomicOrder.acquire,
+    AtomicOrder.seq_cst,
+]
+
+FENCE_ORDERS = [
+    AtomicOrder.acquire,
+    AtomicOrder.release,
+    AtomicOrder.acq_rel,
+    AtomicOrder.seq_cst,
+]
+
+
+class Feature(enum.Flag):
+    v8a = enum.auto()
+    v8_1a = enum.auto()  # -mattr=+v8.1a, mandatory FEAT_LOR, FEAT_LSE
+    rcpc = enum.auto()  # FEAT_LRCPC
+    lse2 = enum.auto()  # FEAT_LSE2
+    outline_atomics = enum.auto()  # -moutline-atomics
+    rcpc3 = enum.auto()  # FEAT_LRCPC3
+    lse128 = enum.auto()  # FEAT_LSE128
+
+    @property
+    def mattr(self):
+        if self == Feature.outline_atomics:
+            return 'outline-atomics'
+        if self == Feature.v8_1a:
+            return 'v8.1a'
+        return self.name
+
+
+ATOMICRMW_OPS = [
+    'xchg',
+    'add',
+    'sub',
+    'and',
+    'nand',
+    'or',
+    'xor',
+    'max',
+    'min',
+    'umax',
+    'umin',
+]
+
+
+def all_atomicrmw(f):
+    for op in ATOMICRMW_OPS:
+        for aligned in Aligned:
+            for ty in Type:
+                for ordering in ATOMICRMW_ORDERS:
+                    name = f'atomicrmw_{op}_{ty}_{aligned}_{ordering}'
+                    instr = 'atomicrmw'
+                    f.write(
+                        textwrap.dedent(f'''
+                        define dso_local {ty} @{name}(ptr %ptr, {ty} %value) {{
+                            %r = {instr} {op} ptr %ptr, {ty} %value {ordering}, align {ty.align(aligned)}
+                            ret {ty} %r
+                        }}
+                    '''))
+
+
+def all_load(f):
+    for aligned in Aligned:
+        for ty in Type:
+            for ordering in ATOMIC_LOAD_ORDERS:
+                for const in [False, True]:
+                    name = f'load_atomic_{ty}_{aligned}_{ordering}'
+                    instr = 'load atomic'
+                    if const:
+                        name += '_const'
+                    arg = 'ptr readonly %ptr' if const else 'ptr %ptr'
+                    f.write(
+                        textwrap.dedent(f'''
+                        define dso_local {ty} @{name}({arg}) {{
+                            %r = {instr} {ty}, ptr %ptr {ordering}, align {ty.align(aligned)}
+                            ret {ty} %r
+                        }}
+                    '''))
+
+
+def all_store(f):
+    for aligned in Aligned:
+        for ty in Type:
+            for ordering in ATOMIC_STORE_ORDERS:  # FIXME stores
+                name = f'store_atomic_{ty}_{aligned}_{ordering}'
+                instr = 'store atomic'
+                f.write(
+                    textwrap.dedent(f'''
+                    define dso_local void @{name}({ty} %value, ptr %ptr) {{
+                        {instr} {ty} %value, ptr %ptr {ordering}, align {ty.align(aligned)}
+                        ret void
+                    }}
+                '''))
+
+
+def all_cmpxchg(f):
+    for aligned in Aligned:
+        for ty in Type:
+            for success_ordering in CMPXCHG_SUCCESS_ORDERS:
+                for failure_ordering in CMPXCHG_FAILURE_ORDERS:
+                    for weak in [False, True]:
+                        name = f'cmpxchg_{ty}_{aligned}_{success_ordering}_{failure_ordering}'
+                        instr = 'cmpxchg'
+                        if weak:
+                            name += '_weak'
+                            instr += ' weak'
+                        f.write(
+                            textwrap.dedent(f'''
+                            define dso_local {ty} @{name}({ty} %expected, {ty} %new, ptr %ptr) {{
+                                %pair = {instr} ptr %ptr, {ty} %expected, {ty} %new {success_ordering} {failure_ordering}, align {ty.align(aligned)}
+                                %r = extractvalue {{ {ty}, i1 }} %pair, 0
+                                ret {ty} %r
+                            }}
+                        '''))
+
+
+def all_fence(f):
+    for ordering in FENCE_ORDERS:
+        name = f'fence_{ordering}'
+        f.write(
+            textwrap.dedent(f'''
+            define dso_local void @{name}() {{
+                fence {ordering}
+                ret void
+            }}
+        '''))
+
+
+def header(f, triple, features, filter_args: str):
+    f.write('; NOTE: Assertions have been autogenerated by '
+            'utils/update_llc_test_checks.py UTC_ARGS: ')
+    f.write(filter_args)
+    f.write('\n')
+    f.write(f'; The base test file was generated by {__file__}\n')
+    for feat in features:
+        for OptFlag in ['-O0', '-O1']:
+            f.write(' '.join([
+                ';', 'RUN:', 'llc', '%s', '-o', '-', '-verify-machineinstrs',
+                f'-mtriple={triple}', f'-mattr=+{feat.mattr}', OptFlag, '|',
+                'FileCheck', '%s', f'--check-prefixes=CHECK,{OptFlag}\n'
+            ]))
+
+
+def write_lit_tests():
+    os.chdir('llvm/test/CodeGen/AArch64/Atomics/')
+    for triple in TRIPLES:
+        # Feature has no effect on fence, so keep it to one file.
+        with open(f'{triple}-fence.ll', 'w') as f:
+            filter_args = r'--filter "^\s*(dmb)"'
+            header(f, triple, Feature, filter_args)
+            all_fence(f)
+
+        for feat in Feature:
+            with open(f'{triple}-atomicrmw-{feat.name}.ll', 'w') as f:
+                filter_args = r'--filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"'
+                header(f, triple, [feat], filter_args)
+                all_atomicrmw(f)
+
+            with open(f'{triple}-cmpxchg-{feat.name}.ll', 'w') as f:
+                filter_args = r'--filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"'
+                header(f, triple, [feat], filter_args)
+                all_cmpxchg(f)
+
+            with open(f'{triple}-atomic-load-{feat.name}.ll', 'w') as f:
+                filter_args = r'--filter-out "\b(sp)\b" --filter "^\s*(ld|st[^r]|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"'
+                header(f, triple, [feat], filter_args)
+                all_load(f)
+
+            with open(f'{triple}-atomic-store-{feat.name}.ll', 'w') as f:
+                filter_args = r'--filter-out "\b(sp)\b" --filter "^\s*(ld[^r]|st|swp|cas|bl|add|and|eor|orn|orr|sub|mvn|sxt|cmp|ccmp|csel|dmb)"'
+                header(f, triple, [feat], filter_args)
+                all_store(f)
+
+if __name__ == '__main__':
+    write_lit_tests()
+
+    print(textwrap.dedent('''
+        Testcases written. To update checks run:
+            $ ./llvm/utils/update_llc_test_checks.py -u llvm/test/CodeGen/AArch64/Atomics/*.ll
+
+        Or in parallel:
+            $ parallel ./llvm/utils/update_llc_test_checks.py -u ::: llvm/test/CodeGen/AArch64/Atomics/*.ll
+    '''))